-- Logs begin at Sun 2022-10-09 00:10:45 UTC, end at Sun 2022-10-09 01:15:45 UTC. -- Oct 09 00:47:13.391878 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 09 00:47:13.400539 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 9 00:47:13 2022] *** Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: nodename: dv-cinder-master-3-iscsi Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: machine: x86_64 Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: clock source: unix Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: pcre jit disabled Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: detected number of CPU cores: 4 Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: current working directory: / Oct 09 00:47:13.402479 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: detected binary path: /usr/bin/uwsgi-core Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: your processes number limit is 47680 Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: your memory page size is 4096 bytes Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: detected max file descriptor number: 2048 Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: lock engine: pthread robust mutexes Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: thunder lock: enabled Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 09 00:47:13.403140 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 09 00:47:13.441997 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: Python main interpreter initialized at 0x558947de89c0 Oct 09 00:47:13.441997 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: python threads support enabled Oct 09 00:47:13.441997 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: your server socket listen backlog is limited to 100 connections Oct 09 00:47:13.441997 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: your mercy for graceful operations on workers is 90 seconds Oct 09 00:47:13.442309 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: mapped 403077 bytes (393 KB) for 2 cores Oct 09 00:47:13.442409 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: *** Operational MODE: preforking *** Oct 09 00:47:13.442666 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: *** uWSGI is running in multiple interpreter mode *** Oct 09 00:47:13.442666 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: spawned uWSGI master process (pid: 105435) Oct 09 00:47:13.442843 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 09 00:47:13.443370 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: spawned uWSGI worker 1 (pid: 105436, cores: 1) Oct 09 00:47:13.443715 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: spawned uWSGI worker 2 (pid: 105437, cores: 1) Oct 09 00:47:13.443875 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 09 00:47:14.099516 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 09 00:47:14.101086 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 09 00:47:15.183952 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:47:15.183952 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: from cryptography.utils import int_from_bytes Oct 09 00:47:15.184635 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:47:15.184635 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: from cryptography.utils import int_from_bytes Oct 09 00:47:15.185746 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:47:15.185746 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: from cryptography.utils import int_from_bytes Oct 09 00:47:15.186166 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:47:15.186166 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: from cryptography.utils import int_from_bytes Oct 09 00:47:17.667077 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 09 00:47:17.667664 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 09 00:47:17.667865 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 09 00:47:17.668212 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 09 00:47:17.668463 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 09 00:47:17.668847 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.669166 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.669261 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 09 00:47:17.669620 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.669715 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 09 00:47:17.669801 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.670028 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 09 00:47:17.670124 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.670378 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.670482 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 09 00:47:17.670628 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 09 00:47:17.670728 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.670988 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.671081 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.671851 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.671963 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.672106 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.672205 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.672412 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.672508 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.672730 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.672823 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673037 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673138 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673341 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673442 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673640 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673725 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.673951 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674045 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674246 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674338 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674557 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674658 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674853 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.674939 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.675184 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.675345 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.675489 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.676913 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.677137 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.677337 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.677489 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.677709 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.677918 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.678111 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.678306 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.678490 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.678691 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.678875 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.679070 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.679285 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.679481 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.679678 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.679890 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.680049 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.680276 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.680472 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.680662 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.680956 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.681154 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.681341 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.681496 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.681726 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.681886 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682063 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682220 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682362 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682496 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682646 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682770 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.682909 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683047 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683178 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683356 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683499 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683666 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683800 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.683929 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684065 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684195 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684325 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684459 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684746 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.684879 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685008 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685143 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685273 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685397 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685534 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685665 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685665 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.685903 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.686032 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.686164 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.686299 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.686433 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.686558 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.686689 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.687455 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.687613 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.687753 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.687893 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688030 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688162 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688299 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688447 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688582 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688721 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688845 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.688976 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689114 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689261 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689392 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689525 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689655 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689793 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.689928 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690060 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690190 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690321 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690445 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690572 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690705 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690829 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.690952 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691073 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691202 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691333 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691464 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691586 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691748 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.691884 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692013 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692152 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692276 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692398 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692532 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692662 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692786 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692920 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.692920 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693151 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693151 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693380 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693508 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693640 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693763 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.693886 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694014 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694149 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694282 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694414 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694543 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694673 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694802 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.694939 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695079 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695202 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695335 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695468 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695611 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695750 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.695888 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696019 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696149 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696278 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696407 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696530 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696664 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696794 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.696922 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697045 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697176 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697312 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697437 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697568 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697698 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697828 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.697957 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698090 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698220 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698351 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698481 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698604 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698726 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698857 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.698981 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699104 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699242 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699374 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699504 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699682 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699809 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.699933 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700062 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700186 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700308 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700442 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700567 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700698 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700821 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.700952 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701084 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701209 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701346 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701477 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701615 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701749 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.701881 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702050 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702188 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702369 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702498 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702691 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702844 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.702994 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.703129 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.703295 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.703424 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.703628 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.703841 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.703976 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704101 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704246 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704386 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704551 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704690 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704856 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.704985 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.705155 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.705288 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.705462 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.705605 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.705746 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.705884 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706054 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706190 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706351 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706486 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706660 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706795 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.706958 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707083 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707256 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707393 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707552 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707711 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707858 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.707989 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.708158 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.708302 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.708455 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.708584 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.708747 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.708875 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709043 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709179 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709334 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709461 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709634 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709767 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.709931 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710058 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710229 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710356 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710525 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710653 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710822 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.710949 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.711122 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.711250 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.711421 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.711562 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.711762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.711885 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.712044 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.712165 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.712348 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.712488 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.712651 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.712787 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.712955 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.713082 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.713256 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.713391 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.713571 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:47:17.713699 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.713900 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.714037 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.714219 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.714353 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.714524 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.714660 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.714863 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715005 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715194 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715331 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715673 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715861 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.715989 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.716170 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.716304 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.716481 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.716616 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.716789 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.716945 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.717103 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.717239 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.717409 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.717552 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.717719 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.717847 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.718057 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.718198 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.718371 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.718498 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.718676 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.718817 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719005 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719149 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719319 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719438 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719654 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719785 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.719968 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.720127 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.720282 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.720416 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.720588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.720724 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.720893 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721022 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721196 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721324 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721531 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721667 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721836 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.721972 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.722144 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.722271 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.722455 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.722604 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.722761 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.722897 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.723076 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.723205 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.723380 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.723514 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.723708 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.723837 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724012 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724139 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724318 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724447 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724622 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724758 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.724953 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.725081 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.725282 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.725419 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.725570 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.725697 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.725879 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726014 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726181 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726308 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726617 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726794 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.726928 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.727097 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.727233 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.727401 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.727537 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.727723 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.727872 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728021 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728148 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728319 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728455 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728622 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728770 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.728904 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729029 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729218 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729345 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729539 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729684 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729843 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.729979 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.730148 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.730281 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.730450 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.730589 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.730754 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.730882 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.731059 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.731193 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.731359 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.731493 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.731699 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.731828 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732027 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732154 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732333 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732468 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732629 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732755 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.732948 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.733091 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.733254 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.733396 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.733541 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.734255 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.734384 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.734507 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.734676 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.734813 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.734976 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.735105 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.735287 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.735422 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.735607 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.735750 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.735923 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736049 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736232 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736536 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736672 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736851 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.736978 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.737153 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.737281 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.737458 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.737601 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.737771 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.737901 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.738082 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.738225 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.738381 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.738524 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.738693 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.738828 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739002 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739148 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739308 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739435 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739644 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739784 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.739934 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740061 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740219 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740355 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740516 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740650 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740798 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.740923 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741102 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741239 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741396 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741532 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741694 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741822 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.741991 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.742128 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.742288 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.742416 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.742584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.742713 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.742900 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743200 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743345 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743496 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743646 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743817 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.743966 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.744107 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.744245 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.744405 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.744533 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.744710 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.744846 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745018 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745156 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745339 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745475 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745644 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745772 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.745945 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746093 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746248 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746377 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746547 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746682 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746848 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.746984 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.747153 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.747289 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.747457 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.747655 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.747799 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.747932 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.748084 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.748220 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.748390 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.748518 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.748702 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.748824 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.749006 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.749133 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.749937 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.750249 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.750564 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.750761 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.750898 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.751095 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.751222 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.751417 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.751555 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.751761 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.751891 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.752092 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.752229 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.752410 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.752541 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.752713 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.752840 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753152 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753327 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753457 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753631 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753758 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.753935 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754062 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754237 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754365 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754549 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754678 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754852 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.754980 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.755182 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.755308 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.755456 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.755584 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.755795 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.755940 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.756104 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.756240 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.756414 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.756550 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.756711 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.756881 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757013 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757143 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757375 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757500 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757623 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757755 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.757940 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758067 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758243 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758375 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758557 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758694 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758861 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.758997 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.759166 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.759293 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.759470 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.759637 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.759793 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.759922 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.760092 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.760221 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.760396 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.760532 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.760704 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.760832 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761007 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761134 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761312 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761447 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761609 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761753 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.761915 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762052 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762216 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762353 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762511 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762648 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762807 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.762942 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.763109 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.763236 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.763413 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.763549 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.763741 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.763892 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764053 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764189 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764362 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764490 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764657 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764793 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.764969 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.765102 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.765282 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.765436 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.765621 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.765761 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.765922 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766059 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766226 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766355 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766525 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766669 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766835 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.766967 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.767150 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.767280 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.767448 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.767575 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.767785 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.767925 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.768098 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.768225 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.768403 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.768563 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.768711 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.768839 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769017 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769152 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769319 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769448 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769625 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769762 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.769928 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770057 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770231 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770359 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770536 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770672 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770842 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.770987 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.771150 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.771280 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.771455 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.771646 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.771773 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.771909 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772086 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772215 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772389 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772525 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772694 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772825 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.773133 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.773296 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.773433 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.773604 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.773740 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.773914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.774169 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.774297 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.774499 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.774637 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.774812 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.774946 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.775131 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.775265 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.775436 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.775580 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.775770 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.775917 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.776087 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.776223 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.776390 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.776526 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.776698 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.776828 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.777016 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.777159 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.777321 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.777576 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.777714 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.777907 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778043 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778213 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778349 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778513 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778658 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778811 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.778939 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.779110 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.779321 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.779499 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.779649 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.779788 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.779914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780037 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780157 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780288 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780411 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780657 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780796 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.780973 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.781102 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.781274 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.781410 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.781583 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.781712 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.781883 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782012 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782187 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782323 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782492 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782629 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782790 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.782924 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.783090 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.783226 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.783389 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.783516 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.783711 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.783840 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784014 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784155 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784327 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784457 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784629 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784773 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.784941 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785071 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785242 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785378 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785562 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785697 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785867 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.785995 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.786170 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.786304 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.786467 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.786595 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.786763 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.786900 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787066 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787204 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787367 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787505 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787692 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787828 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.787997 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.788134 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.788300 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.788453 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.788605 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.788734 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.788908 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789037 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789205 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789342 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789509 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789637 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789805 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.789933 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790111 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790256 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790397 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790528 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790696 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790832 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.790989 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.791133 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.791291 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.791421 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.791633 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.791777 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.791912 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792044 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792201 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792346 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792499 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792636 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792802 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.792937 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793098 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793243 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793397 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793535 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793695 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793832 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.793991 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.794126 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.794290 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.794432 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.794573 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.794696 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.794886 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795023 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795181 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795332 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795465 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795608 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795795 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.795940 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.796620 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.796769 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.796930 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797061 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797226 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797354 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797525 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797670 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797834 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.797963 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.798140 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.798269 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.798435 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.798562 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.798737 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.798865 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799033 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799161 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799331 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799459 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799650 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799779 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.799940 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800088 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800239 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800361 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800519 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800650 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800815 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.800931 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.801362 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.801485 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.801659 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.801798 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.801967 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802112 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802267 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802393 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802564 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802706 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802859 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.802995 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.803155 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.803283 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.803457 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.803613 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.803780 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.803931 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804080 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804217 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804377 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804514 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804677 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804815 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.804977 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.805115 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.805295 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.805431 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.805599 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.805729 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.805898 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806028 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806199 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806326 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806496 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806639 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806799 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.806925 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.807108 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.807237 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.807438 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.807573 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.807742 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.807865 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808018 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808148 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808324 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808453 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808628 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808756 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.808944 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809079 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809262 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809399 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809562 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809704 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809863 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.809998 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.810168 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.810296 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.810472 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.810607 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.810771 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.810915 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.811072 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.811209 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.811375 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.811501 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.811705 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.812376 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.812561 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.812688 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.812883 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813012 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813204 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813350 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813510 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813637 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813815 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.813951 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.814120 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.814248 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.814423 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.814558 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.814723 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.814851 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815030 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815164 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815335 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815463 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815664 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815803 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.815979 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.816125 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.816294 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.816431 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.816604 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.816733 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.816914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817051 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817218 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817347 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817529 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817667 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817837 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.817973 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.818141 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.818276 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.818462 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.818609 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.818773 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.818911 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.819075 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.819220 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.819383 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.819520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.819708 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.819847 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.820018 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.820162 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.820323 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.820457 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.820706 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.820834 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821016 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821159 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821318 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821446 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821615 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821744 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.821919 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822055 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822226 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822354 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822524 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822651 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822821 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.822965 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.823119 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.823256 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.823435 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.823563 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.823757 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.823888 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.824074 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.824203 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.824448 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.824584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.824751 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.824902 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825044 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825169 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825347 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825478 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825669 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.825970 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.826107 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.826277 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.826405 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.826577 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.826722 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.826897 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.827033 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.827199 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.827335 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.827540 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.827756 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.827903 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828173 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828297 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828493 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828629 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828800 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.828945 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.829111 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.829240 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.829411 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.829556 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.829728 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.829867 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.830043 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.830182 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.830353 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.830489 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.830668 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.830805 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.831068 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.831205 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.831379 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.831493 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.831706 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.831807 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832026 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832160 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832335 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832471 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832635 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832763 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.832934 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.833063 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.833240 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.833379 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.833537 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.833731 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.833871 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834004 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834127 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834258 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834416 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834545 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834722 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.834859 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835026 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835164 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835331 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835462 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835656 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835798 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.835960 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.836098 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.836278 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.836414 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.836574 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.836704 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.836890 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837019 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837198 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837333 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837514 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837643 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837832 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.837968 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.838131 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.838260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.838438 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.838573 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.838737 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.838866 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839034 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839162 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839330 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839457 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839649 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839773 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.839958 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.840098 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.840276 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.840417 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.840586 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.840714 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.840907 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841036 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841227 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841369 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841532 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841662 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841853 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.841982 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.842160 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.842298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.842462 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.842592 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.842764 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.842900 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.843095 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.843235 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.843405 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.843541 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.843735 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.843875 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844015 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844140 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844324 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844462 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844623 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844752 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.844922 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.845067 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.845222 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.845358 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.845524 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.845661 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.845876 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846013 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846184 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846312 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846481 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846786 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.846913 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.847086 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.847214 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.847403 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.847549 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.847729 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.847866 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848030 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848170 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848314 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848447 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848620 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848742 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.848924 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849069 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849228 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849375 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849531 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849660 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849835 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.849963 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.850151 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.850287 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.850455 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.850583 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.850767 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.850894 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.851077 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.851206 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.851382 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.851527 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.851709 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.851838 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852013 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852144 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852323 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852463 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852632 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852769 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.852944 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.853074 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.853254 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.853399 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.853566 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.853703 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.853910 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.854078 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.854218 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.854401 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.854529 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.854712 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.854841 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855018 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855161 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855329 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855466 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855659 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855789 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.855976 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.856113 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.856285 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.856423 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.856591 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.856721 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.856898 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.857027 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.857206 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.857418 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.857550 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.857746 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.857875 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.858062 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.858197 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.858374 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.858505 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.859213 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.859356 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.859531 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.859713 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.859919 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860047 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860228 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860358 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860538 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860676 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860848 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.860977 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.861157 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.861294 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.861466 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.861668 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.861796 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.861920 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862072 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862209 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862378 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862515 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862683 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862813 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.862991 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.863119 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.863297 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.863426 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.863613 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.863746 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.863927 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.864056 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.864237 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.864368 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.864547 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.864684 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.864867 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865008 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865179 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865306 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865494 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865624 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865800 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.865936 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.866112 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.866239 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.866421 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.866555 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.866730 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.866929 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.867076 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.867239 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.867375 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.867567 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.867725 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.867911 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868042 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868231 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868359 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868543 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868681 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868853 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.868982 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.869175 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.869311 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.869474 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.869624 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.869772 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.869901 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.870078 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.870290 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.870426 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.870604 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.870735 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.870936 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.871075 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.871261 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.871404 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.871544 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.871716 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.871881 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.872010 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.872185 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.872339 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.872523 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.872708 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.872837 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873054 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873183 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873368 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873505 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873692 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873828 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.873990 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.874124 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.874354 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.874967 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.875145 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.875274 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.875449 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.875610 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.875781 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.875909 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.876089 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.876226 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.876400 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.876621 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.876759 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.876973 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.877110 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.877278 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.877405 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.877584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.877714 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.877886 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878023 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878184 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878322 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878489 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878634 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878787 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.878915 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.879109 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.879255 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.879410 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.879555 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.879734 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.879881 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880034 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880169 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880340 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880477 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880640 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880776 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.880937 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881075 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881237 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881374 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881534 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881671 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881840 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.881975 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.882134 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.882261 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.882436 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.882566 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.882733 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.882862 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883037 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883173 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883340 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883469 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883671 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883803 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.883975 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.884104 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.884276 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.884414 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.884576 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.884713 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.884874 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885012 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885183 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885311 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885486 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885613 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885789 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.885916 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.886096 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.886233 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.886398 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.886532 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.886739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.886868 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.887043 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.887179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.887345 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.887566 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.887715 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.887919 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.888056 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.888240 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.888385 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.888564 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.888692 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.888873 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889010 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889177 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889304 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889476 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889612 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889775 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.889971 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.890112 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.890253 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.890412 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.890541 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.890719 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.890962 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.891106 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.891293 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.891422 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.891641 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.891781 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.891927 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892063 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892230 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892367 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892534 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892671 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892837 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.892973 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.893140 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.893276 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.893442 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.893579 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.893744 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.893873 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894047 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894176 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894349 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894495 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894655 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894790 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.894964 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.895093 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.895271 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.895393 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.895578 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.895733 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.895904 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896032 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896217 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896354 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896522 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896698 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896835 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.896958 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.897119 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.897259 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.897423 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.897559 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.897730 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.897866 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898040 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898347 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898482 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898647 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898792 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.898991 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.899301 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.899430 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.899697 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.899818 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.899955 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900081 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900227 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900355 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900536 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900681 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900845 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.900990 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.901159 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.901296 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.901463 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.901599 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.901778 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.901906 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.902080 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.902223 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.902402 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.902734 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.902871 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903054 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903181 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903372 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903499 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903695 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903826 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.903996 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.904124 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.904294 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.904432 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.904597 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.904726 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.904908 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905052 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905211 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905358 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905561 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905708 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905833 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.905956 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.906124 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.906266 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.906415 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.906549 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.906708 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.906836 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907005 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907150 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907303 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907440 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907623 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907766 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.907906 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908032 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908207 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908335 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908505 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908634 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908801 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.908938 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909096 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909235 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909393 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909521 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909695 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909826 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.909991 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.910119 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.910289 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.910434 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.910586 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.910714 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.910882 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911012 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911175 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911320 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911500 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911643 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911830 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.911959 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.912133 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.912283 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.912417 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.912541 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.912722 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.912860 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913028 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913166 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913329 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913465 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913631 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913768 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.913932 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914071 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914233 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914362 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914534 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914670 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914860 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.914988 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.915167 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.915303 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.915475 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.915641 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.915813 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.915944 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.916127 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.916270 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.916416 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.916540 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.916743 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.916880 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917064 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917237 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917371 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917507 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917686 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917816 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.917996 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.918141 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.918313 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.918449 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.918620 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.918766 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.918932 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919061 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919237 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919453 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919581 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919725 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919849 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.919981 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.920160 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.920290 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.920477 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.920604 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.920784 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.920913 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.921091 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.921741 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.921927 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922070 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922235 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922364 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922543 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922672 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922855 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.922990 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.923164 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.923300 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.923484 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.923672 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.923812 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.923938 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.924115 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.924250 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.924425 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.924556 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.924735 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.924865 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925047 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925185 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925362 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925491 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925672 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925809 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.925984 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.926359 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.926554 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.926685 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.926861 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.926988 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.927164 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.927300 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.927433 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105437) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 09 00:47:17.927565 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.927818 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.928142 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.928467 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.928812 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.929126 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.929471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.929798 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.930111 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.930423 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.930727 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:47:17.930969 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105436) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 09 00:47:18.080015 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO dbcounter [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Registered counter for database nova_api Oct 09 00:47:18.083005 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO dbcounter [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Registered counter for database nova_api Oct 09 00:47:18.100490 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_db.sqlalchemy.engines [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.102971 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_db.sqlalchemy.engines [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.107267 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writer thread running {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.109924 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writer thread running {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.165033 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.context [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8734a8b3-2f61-4ffd-a611-f81fb26e2dd6(cell1) {{(pid=105437) load_cells /opt/stack/nova/nova/context.py:464}} Oct 09 00:47:18.165571 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.context [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8734a8b3-2f61-4ffd-a611-f81fb26e2dd6(cell1) {{(pid=105436) load_cells /opt/stack/nova/nova/context.py:464}} Oct 09 00:47:18.168992 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_concurrency.lockutils [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105437) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:47:18.169407 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:47:18.169512 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_concurrency.lockutils [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105437) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:47:18.169879 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:47:18.170981 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_concurrency.lockutils [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105437) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:47:18.172040 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:47:18.172342 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_concurrency.lockutils [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105437) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:47:18.172762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:47:18.172871 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_concurrency.lockutils [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105437) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:47:18.173201 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:47:18.174738 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_concurrency.lockutils [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105437) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:47:18.175176 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:47:18.176997 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO dbcounter [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Registered counter for database nova_cell0 Oct 09 00:47:18.177424 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO dbcounter [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Registered counter for database nova_cell0 Oct 09 00:47:18.181053 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO dbcounter [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Registered counter for database nova_cell1 Oct 09 00:47:18.181365 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO dbcounter [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Registered counter for database nova_cell1 Oct 09 00:47:18.188239 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_db.sqlalchemy.engines [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.189832 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_db.sqlalchemy.engines [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.190775 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_db.sqlalchemy.engines [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.193030 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writer thread running {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.193663 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_db.sqlalchemy.engines [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.196741 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writer thread running {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.199972 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writer thread running {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.333810 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writer thread running {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.341472 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO dbcounter [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Registered counter for database nova_cell0 Oct 09 00:47:18.342562 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO dbcounter [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Registered counter for database nova_cell0 Oct 09 00:47:18.349171 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_db.sqlalchemy.engines [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105436) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.352796 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG oslo_db.sqlalchemy.engines [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105437) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:47:18.354020 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writer thread running {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.357408 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writer thread running {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:47:18.690771 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: WARNING keystonemiddleware.auth_token [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:47:18.691542 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: WARNING keystonemiddleware.auth_token [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:47:18.698019 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: WARNING keystonemiddleware.auth_token [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:47:18.698716 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: WARNING keystonemiddleware.auth_token [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:47:19.054968 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: WARNING keystonemiddleware.auth_token [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:47:19.060864 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: WARNING keystonemiddleware.auth_token [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:47:19.063081 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: WARNING keystonemiddleware.auth_token [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:47:19.066832 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558947de89c0 pid: 105436 (default app) Oct 09 00:47:19.068891 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:47:19.069000 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: WARNING keystonemiddleware.auth_token [None req-b63a0e4e-505a-4eb9-b3b7-10b7a924c7a6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:47:19.070072 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] 10.222.0.33 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001706 Oct 09 00:47:19.070487 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 788 bytes} [Sun Oct 9 00:47:19 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:47:19.074350 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558947de89c0 pid: 105437 (default app) Oct 09 00:48:39.812771 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:39.813773 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001454 Oct 09 00:48:39.814170 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 2/2] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:48:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:48:39.826832 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_api:SELECT=3 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:39.827907 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_cell1:SELECT=3 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:39.828976 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_cell0:SELECT=3 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:39.830905 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:40.707994 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-026e606d-ab22-48ac-9c2e-64f238a12c1e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:40.741414 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-026e606d-ab22-48ac-9c2e-64f238a12c1e admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.919877 Oct 09 00:48:40.742152 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 1/3] 10.222.0.33 () {60 vars in 1292 bytes} [Sun Oct 9 00:48:39 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 922 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:48:43.348276 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:43.349290 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-b5523fe0-80a5-4817-8b6b-824071c810c9 None None] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Oct 09 00:48:43.351701 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 3/4] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:48:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:48:43.794342 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-ebccf574-803c-4214-8c09-105003e100b6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 256, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "c1", "vcpus": 1, "name": "cirros256"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:43.841040 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-ebccf574-803c-4214-8c09-105003e100b6 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.485023 Oct 09 00:48:43.841602 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 2/5] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:48:43 2022] POST /compute/v2.1/flavors => generated 407 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:48:43.855425 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_api:SELECT=3 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:43.856271 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_cell1:SELECT=3 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:43.858181 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_cell0:SELECT=3 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:43.858913 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:48:44.064927 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-1091520b-b625-4b60-9886-1e7f774986d1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:44.139537 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-1091520b-b625-4b60-9886-1e7f774986d1 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.289156 Oct 09 00:48:44.140188 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 4/6] 10.222.0.33 () {64 vars in 1336 bytes} [Sun Oct 9 00:48:43 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:48:46.757176 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-ebccf574-803c-4214-8c09-105003e100b6 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:46.758184 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-ebccf574-803c-4214-8c09-105003e100b6 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001454 Oct 09 00:48:46.758580 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 3/7] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:48:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:48:47.419533 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-5eb8bea3-4e3e-4d33-83d7-85ef6bf7013c admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "name": "ds512M", "vcpus": 1, "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 5}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:47.461282 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-5eb8bea3-4e3e-4d33-83d7-85ef6bf7013c admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.696302 Oct 09 00:48:47.463657 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 5/8] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:48:46 2022] POST /compute/v2.1/flavors => generated 404 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 00:48:47.474161 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-e1588493-b79d-4b68-a068-f42dcc69e8e7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:47.521634 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-e1588493-b79d-4b68-a068-f42dcc69e8e7 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.051133 Oct 09 00:48:47.522640 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 4/9] 10.222.0.33 () {64 vars in 1336 bytes} [Sun Oct 9 00:48:47 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:48:50.144477 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-5eb8bea3-4e3e-4d33-83d7-85ef6bf7013c admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:50.146245 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-5eb8bea3-4e3e-4d33-83d7-85ef6bf7013c admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002200 Oct 09 00:48:50.146844 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 6/10] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:48:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:48:50.580809 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-3eb8113a-2d3c-438f-8a46-b0364b71f153 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "rxtx_factor": 1.0, "id": "d2", "os-flavor-access:is_public": true, "swap": 0, "name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "disk": 10}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:50.600121 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-3eb8113a-2d3c-438f-8a46-b0364b71f153 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.446861 Oct 09 00:48:50.600670 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 5/11] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:48:50 2022] POST /compute/v2.1/flavors => generated 404 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:48:50.612854 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-cbae4c62-0d81-4616-8ed1-e88ce6e50a76 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:50.644479 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-cbae4c62-0d81-4616-8ed1-e88ce6e50a76 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035326 Oct 09 00:48:50.645238 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 7/12] 10.222.0.33 () {64 vars in 1336 bytes} [Sun Oct 9 00:48:50 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:48:53.270207 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-3eb8113a-2d3c-438f-8a46-b0364b71f153 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:53.271219 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-3eb8113a-2d3c-438f-8a46-b0364b71f153 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 Oct 09 00:48:53.273249 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 6/13] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:48:53 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:48:53.684596 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-710ac005-e48d-4df1-a8f5-e1570eb85224 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 10, "swap": 0, "vcpus": 2, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "ds2G", "id": "d3", "ram": 2048}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:53.704444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-710ac005-e48d-4df1-a8f5-e1570eb85224 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.426644 Oct 09 00:48:53.705384 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 8/14] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:48:53 2022] POST /compute/v2.1/flavors => generated 404 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:48:53.717911 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-c2acce3a-9b53-4140-add8-833be1ac7e1f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:53.745144 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-c2acce3a-9b53-4140-add8-833be1ac7e1f admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030883 Oct 09 00:48:53.745712 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 7/15] 10.222.0.33 () {64 vars in 1336 bytes} [Sun Oct 9 00:48:53 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:48:56.368706 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-710ac005-e48d-4df1-a8f5-e1570eb85224 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:48:56.370004 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-710ac005-e48d-4df1-a8f5-e1570eb85224 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001941 Oct 09 00:48:56.370559 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 9/16] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:48:56 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:48:56.790433 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-4b22848e-6702-462b-ad68-532bb4b69db4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "disk": 20, "vcpus": 4, "swap": 0, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds4G"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:56.813764 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-4b22848e-6702-462b-ad68-532bb4b69db4 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.436710 Oct 09 00:48:56.814599 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 8/17] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:48:56 2022] POST /compute/v2.1/flavors => generated 404 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 00:48:56.829433 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-79966825-ad3e-4df5-a60b-e3be135a552a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:48:56.854761 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-79966825-ad3e-4df5-a60b-e3be135a552a admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028992 Oct 09 00:48:56.855345 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 10/18] 10.222.0.33 () {64 vars in 1336 bytes} [Sun Oct 9 00:48:56 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:49:00.238176 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-4b22848e-6702-462b-ad68-532bb4b69db4 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:00.239558 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-4b22848e-6702-462b-ad68-532bb4b69db4 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001857 Oct 09 00:49:00.240026 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 9/19] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:49:00.666116 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-e4cde248-36e4-42ed-8aa0-8b7bb2d12e1a admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:00.698924 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-e4cde248-36e4-42ed-8aa0-8b7bb2d12e1a admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.452546 Oct 09 00:49:00.699708 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 11/20] 10.222.0.33 () {60 vars in 1292 bytes} [Sun Oct 9 00:49:00 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 00:49:03.262353 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-4b22848e-6702-462b-ad68-532bb4b69db4 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:03.263369 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-4b22848e-6702-462b-ad68-532bb4b69db4 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001461 Oct 09 00:49:03.263803 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 10/21] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:03.673857 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-7aac15e3-131f-4944-823e-f0f3f4fc0e82 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "id": "1", "name": "m1.tiny", "ram": 512, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 1}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:03.694260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-7aac15e3-131f-4944-823e-f0f3f4fc0e82 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.424483 Oct 09 00:49:03.694818 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 12/22] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:49:03 2022] POST /compute/v2.1/flavors => generated 402 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 00:49:03.707146 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-35ab6862-f054-40d4-af4c-a3b7dc9b67c2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:03.735970 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-35ab6862-f054-40d4-af4c-a3b7dc9b67c2 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032402 Oct 09 00:49:03.736566 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 11/23] 10.222.0.33 () {64 vars in 1333 bytes} [Sun Oct 9 00:49:03 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:49:06.317411 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-7aac15e3-131f-4944-823e-f0f3f4fc0e82 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:06.318431 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-7aac15e3-131f-4944-823e-f0f3f4fc0e82 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001541 Oct 09 00:49:06.319037 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:06.713673 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-d8b58287-f20e-455f-a934-a22c0c459f01 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "vcpus": 1, "name": "m1.small", "id": "2", "disk": 20, "os-flavor-access:is_public": true, "rxtx_factor": 1.0}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:06.733122 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-d8b58287-f20e-455f-a934-a22c0c459f01 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.408410 Oct 09 00:49:06.733703 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 12/25] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:49:06 2022] POST /compute/v2.1/flavors => generated 405 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:49:06.745502 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2d4f7246-99a4-430b-9b26-820ebbaac53d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:06.769684 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-2d4f7246-99a4-430b-9b26-820ebbaac53d admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027711 Oct 09 00:49:06.770267 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 14/26] 10.222.0.33 () {64 vars in 1333 bytes} [Sun Oct 9 00:49:06 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:49:09.371613 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-d8b58287-f20e-455f-a934-a22c0c459f01 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:09.372736 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-d8b58287-f20e-455f-a934-a22c0c459f01 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001630 Oct 09 00:49:09.373211 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 13/27] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:09 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:09.778669 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-4811acbc-b712-44be-9dcc-61e5db08e3bc admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "rxtx_factor": 1.0, "ram": 4096, "swap": 0, "disk": 40, "os-flavor-access:is_public": true, "id": "3", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:09.803857 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-4811acbc-b712-44be-9dcc-61e5db08e3bc admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.424206 Oct 09 00:49:09.804341 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 15/28] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:49:09 2022] POST /compute/v2.1/flavors => generated 406 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:49:09.817042 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-009f88c8-43d9-4b98-8105-4338e3575208 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:09.857164 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-009f88c8-43d9-4b98-8105-4338e3575208 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.044013 Oct 09 00:49:09.858460 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 14/29] 10.222.0.33 () {64 vars in 1333 bytes} [Sun Oct 9 00:49:09 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:49:12.514118 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-4811acbc-b712-44be-9dcc-61e5db08e3bc admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:12.515829 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-4811acbc-b712-44be-9dcc-61e5db08e3bc admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002108 Oct 09 00:49:12.521752 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 16/30] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:12 2022] GET /compute/v2.1 => generated 388 bytes in 9 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:12.961961 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-59e5dfe1-d311-449a-9ba9-f6c4b9b5b211 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "os-flavor-access:is_public": true, "name": "m1.large", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "4", "ram": 8192, "rxtx_factor": 1.0, "swap": 0, "vcpus": 4}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:12.967794 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:49:12.984584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-59e5dfe1-d311-449a-9ba9-f6c4b9b5b211 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.461548 Oct 09 00:49:12.985137 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 15/31] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:49:12 2022] POST /compute/v2.1/flavors => generated 405 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 00:49:12.997775 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-ad35e9c9-c9a1-4305-b3c9-7f8bf8c0a68f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:13.022840 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-ad35e9c9-c9a1-4305-b3c9-7f8bf8c0a68f admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028666 Oct 09 00:49:13.023415 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 17/32] 10.222.0.33 () {64 vars in 1333 bytes} [Sun Oct 9 00:49:12 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:49:15.621550 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-59e5dfe1-d311-449a-9ba9-f6c4b9b5b211 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:15.622479 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-59e5dfe1-d311-449a-9ba9-f6c4b9b5b211 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001520 Oct 09 00:49:15.622887 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 16/33] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:16.121608 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-f22eb098-3f3a-4d34-8eeb-d55438044640 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "disk": 160, "rxtx_factor": 1.0, "name": "m1.xlarge", "vcpus": 8, "id": "5", "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:16.135347 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:49:16.150969 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-f22eb098-3f3a-4d34-8eeb-d55438044640 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.521962 Oct 09 00:49:16.157193 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 18/34] 10.222.0.33 () {64 vars in 1283 bytes} [Sun Oct 9 00:49:15 2022] POST /compute/v2.1/flavors => generated 408 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 00:49:16.164373 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-b9dc113c-2ce3-4e3f-9381-c4d98eb63004 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:16.189265 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-b9dc113c-2ce3-4e3f-9381-c4d98eb63004 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029282 Oct 09 00:49:16.189758 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 17/35] 10.222.0.33 () {64 vars in 1333 bytes} [Sun Oct 9 00:49:16 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:49:35.721758 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-f22eb098-3f3a-4d34-8eeb-d55438044640 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:35.722768 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-f22eb098-3f3a-4d34-8eeb-d55438044640 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Oct 09 00:49:35.723200 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 19/36] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:49:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:49:35.733811 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:49:36.113175 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-1ef2e7dd-55ff-4c51-b9b4-2df676b5721d admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:36.132543 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-1ef2e7dd-55ff-4c51-b9b4-2df676b5721d admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.403212 Oct 09 00:49:36.133081 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 18/37] 10.222.0.33 () {60 vars in 1292 bytes} [Sun Oct 9 00:49:35 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:49:37.427985 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-f22eb098-3f3a-4d34-8eeb-d55438044640 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:37.428993 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-f22eb098-3f3a-4d34-8eeb-d55438044640 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Oct 09 00:49:37.429449 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 20/38] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:37.956051 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-7249d655-b6cd-4003-bac1-348964a1dfd1 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:37.961330 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.wsgi [None req-7249d655-b6cd-4003-bac1-348964a1dfd1 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-7249d655-b6cd-4003-bac1-348964a1dfd1 admin admin] Request method failure captured: Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Accept: application/json Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Accept-Encoding: gzip, deflate Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Connection: keep-alive Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Content-Length: 0 Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Host: 10.222.0.33 Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Openstack-Api-Version: compute 2.92 Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Openstack-System-Scope: None Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: User-Agent: python-novaclient Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Auth-Token: gAAAAABjQhqhAFB1X4V8Hul58FvgISwCV7C9t4OtNHp88t_ph0z7uTSuxeZPHL0zbXSf8CfkqBkS1bkI4g0Vrmu-z2Fc-a-lySmYAvZsVQ3ff7Up4zbwcEFth-3VV1GmTNze800DDPsECfS6ut-HIfJUicnLJMCLHTYHo-8gXw1ficBQQ43h7dg Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Domain-Id: None Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Domain-Name: None Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Identity-Status: Confirmed Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Is-Admin-Project: True Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Openstack-Nova-Api-Version: 2.92 Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Domain-Id: default Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Domain-Name: Default Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Name: admin Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Role: member,admin,reader Oct 09 00:49:37.964588 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Roles: member,admin,reader Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant: admin Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant-Name: admin Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User: admin Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Domain-Id: default Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Domain-Name: Default Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Id: d9c8218a78034540ba9ed09b697588a8 Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Name: admin Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: method: > Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: exception: Flavor 42 could not be found. Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: action_args: {'id': '42'} Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:49:37.966203 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:37.967113 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:37.968298 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-7249d655-b6cd-4003-bac1-348964a1dfd1 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105437) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:49:37.969941 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-7249d655-b6cd-4003-bac1-348964a1dfd1 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.394933 Oct 09 00:49:37.969941 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 19/39] 10.222.0.33 () {60 vars in 1190 bytes} [Sun Oct 9 00:49:37 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 396 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 09 00:49:37.976954 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-bc74e4fb-aa34-4ae8-8735-b44267295e6b admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:37.999580 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-bc74e4fb-aa34-4ae8-8735-b44267295e6b admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026782 Oct 09 00:49:38.000126 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 21/40] 10.222.0.33 () {60 vars in 1210 bytes} [Sun Oct 9 00:49:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:49:38.010845 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-645a7fce-aec1-47be-b9bb-6db37964f3d8 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:38.033355 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-645a7fce-aec1-47be-b9bb-6db37964f3d8 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026964 Oct 09 00:49:38.033924 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 20/41] 10.222.0.33 () {60 vars in 1210 bytes} [Sun Oct 9 00:49:38 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:49:38.043205 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2ea6b30e-a704-4657-905a-2944cf75d0f4 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:38.049118 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.wsgi [None req-2ea6b30e-a704-4657-905a-2944cf75d0f4 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2ea6b30e-a704-4657-905a-2944cf75d0f4 admin admin] Request method failure captured: Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Accept: application/json Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Accept-Encoding: gzip, deflate Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Connection: keep-alive Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Content-Length: 0 Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Host: 10.222.0.33 Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Openstack-Api-Version: compute 2.92 Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Openstack-System-Scope: None Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: User-Agent: python-novaclient Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Auth-Token: gAAAAABjQhqhAFB1X4V8Hul58FvgISwCV7C9t4OtNHp88t_ph0z7uTSuxeZPHL0zbXSf8CfkqBkS1bkI4g0Vrmu-z2Fc-a-lySmYAvZsVQ3ff7Up4zbwcEFth-3VV1GmTNze800DDPsECfS6ut-HIfJUicnLJMCLHTYHo-8gXw1ficBQQ43h7dg Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Domain-Id: None Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Domain-Name: None Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Identity-Status: Confirmed Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Is-Admin-Project: True Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Openstack-Nova-Api-Version: 2.92 Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Domain-Id: default Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Domain-Name: Default Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Name: admin Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Role: member,admin,reader Oct 09 00:49:38.052444 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Roles: member,admin,reader Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant: admin Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant-Name: admin Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User: admin Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Domain-Id: default Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Domain-Name: Default Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Id: d9c8218a78034540ba9ed09b697588a8 Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Name: admin Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: method: > Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: exception: Flavor 42 could not be found. Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: action_args: {'id': '42'} Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:49:38.053914 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:38.054885 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:38.056618 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2ea6b30e-a704-4657-905a-2944cf75d0f4 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105436) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:49:38.057819 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-2ea6b30e-a704-4657-905a-2944cf75d0f4 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014189 Oct 09 00:49:38.057819 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 22/42] 10.222.0.33 () {60 vars in 1190 bytes} [Sun Oct 9 00:49:38 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 09 00:49:39.275647 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-645a7fce-aec1-47be-b9bb-6db37964f3d8 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:39.276668 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-645a7fce-aec1-47be-b9bb-6db37964f3d8 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Oct 09 00:49:39.277071 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 21/43] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:49:39.798925 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-c30180f3-85f5-4e46-b63b-d0134d9d23f6 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:39.803978 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.wsgi [None req-c30180f3-85f5-4e46-b63b-d0134d9d23f6 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-c30180f3-85f5-4e46-b63b-d0134d9d23f6 admin admin] Request method failure captured: Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Accept: application/json Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Accept-Encoding: gzip, deflate Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Connection: keep-alive Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Content-Length: 0 Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Host: 10.222.0.33 Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Openstack-Api-Version: compute 2.92 Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Openstack-System-Scope: None Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: User-Agent: python-novaclient Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Auth-Token: gAAAAABjQhqjLFnJO5Zw69SwEUv2xuHdi5-6VfifDw6FDwQty85goJ5FDJPG7cT0Zt0I_kene_9Ym764qiKlW5lEjw5oFsFDMe0ywavw6CEI4awuzK1ZFtTgKQ3fCg5OeLdr9UfyCigA2AkRknz5pHk2jZEJAPcpHRmhWDNjgzHflZMoPfs1NPA Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Domain-Id: None Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Domain-Name: None Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Identity-Status: Confirmed Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Is-Admin-Project: True Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Openstack-Nova-Api-Version: 2.92 Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Domain-Id: default Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Domain-Name: Default Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Name: admin Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Role: member,admin,reader Oct 09 00:49:39.805471 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Roles: member,admin,reader Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant: admin Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant-Name: admin Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User: admin Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Domain-Id: default Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Domain-Name: Default Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Id: d9c8218a78034540ba9ed09b697588a8 Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Name: admin Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: method: > Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: exception: Flavor 84 could not be found. Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: action_args: {'id': '84'} Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:49:39.807101 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:39.807947 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.809260 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-c30180f3-85f5-4e46-b63b-d0134d9d23f6 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105436) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:49:39.810291 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-c30180f3-85f5-4e46-b63b-d0134d9d23f6 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.392712 Oct 09 00:49:39.810291 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 23/44] 10.222.0.33 () {60 vars in 1190 bytes} [Sun Oct 9 00:49:39 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 394 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 09 00:49:39.816985 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-55273f74-09c9-4526-9f2a-5a543bf85223 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:39.831790 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-55273f74-09c9-4526-9f2a-5a543bf85223 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018471 Oct 09 00:49:39.832311 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 22/45] 10.222.0.33 () {60 vars in 1210 bytes} [Sun Oct 9 00:49:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:49:39.841345 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-7a3d15b7-d4cb-48e8-b241-ba8ec48c61fd admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:39.856976 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-7a3d15b7-d4cb-48e8-b241-ba8ec48c61fd admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019090 Oct 09 00:49:39.857494 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1210 bytes} [Sun Oct 9 00:49:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:49:39.866971 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-86b96bf5-f500-49df-9a3c-3906a2a26061 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:39.871892 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.wsgi [None req-86b96bf5-f500-49df-9a3c-3906a2a26061 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-86b96bf5-f500-49df-9a3c-3906a2a26061 admin admin] Request method failure captured: Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Accept: application/json Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Accept-Encoding: gzip, deflate Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Connection: keep-alive Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Content-Length: 0 Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Host: 10.222.0.33 Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Openstack-Api-Version: compute 2.92 Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Openstack-System-Scope: None Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: User-Agent: python-novaclient Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Auth-Token: gAAAAABjQhqjLFnJO5Zw69SwEUv2xuHdi5-6VfifDw6FDwQty85goJ5FDJPG7cT0Zt0I_kene_9Ym764qiKlW5lEjw5oFsFDMe0ywavw6CEI4awuzK1ZFtTgKQ3fCg5OeLdr9UfyCigA2AkRknz5pHk2jZEJAPcpHRmhWDNjgzHflZMoPfs1NPA Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Domain-Id: None Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Domain-Name: None Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Identity-Status: Confirmed Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Is-Admin-Project: True Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Openstack-Nova-Api-Version: 2.92 Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Domain-Id: default Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Domain-Name: Default Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Name: admin Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Role: member,admin,reader Oct 09 00:49:39.873376 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Roles: member,admin,reader Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant: admin Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant-Name: admin Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User: admin Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Domain-Id: default Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Domain-Name: Default Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Id: d9c8218a78034540ba9ed09b697588a8 Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Name: admin Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: method: > Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: exception: Flavor 84 could not be found. Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: action_args: {'id': '84'} Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:49:39.874806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:39.875739 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:39.877179 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-86b96bf5-f500-49df-9a3c-3906a2a26061 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105437) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:49:39.879145 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-86b96bf5-f500-49df-9a3c-3906a2a26061 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011217 Oct 09 00:49:39.879145 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 23/47] 10.222.0.33 () {60 vars in 1190 bytes} [Sun Oct 9 00:49:39 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 09 00:49:41.087153 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-7a3d15b7-d4cb-48e8-b241-ba8ec48c61fd admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:41.088181 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-7a3d15b7-d4cb-48e8-b241-ba8ec48c61fd admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Oct 09 00:49:41.088570 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 25/48] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:41.610046 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-00e992c4-5538-4606-b301-2832e77a807d admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:41.620030 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.wsgi [None req-00e992c4-5538-4606-b301-2832e77a807d admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-00e992c4-5538-4606-b301-2832e77a807d admin admin] Request method failure captured: Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Accept: application/json Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Accept-Encoding: gzip, deflate Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Connection: keep-alive Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Content-Length: 0 Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Host: 10.222.0.33 Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Openstack-Api-Version: compute 2.92 Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Openstack-System-Scope: None Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: User-Agent: python-novaclient Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Auth-Token: gAAAAABjQhqkao7dd8gAqmm5BJHtI0OgMg_UX_KUmwNJtf_oVsrRiJCmNhkcHCd4X91jWZ-fb8sGne2CMgzG60Al7LdC79UIG_JJ-JeIZADLkYE0YVzLBoBFOUBTnMV4Suv_fDZtk54fk8p8Jc8cAq2M1nbJ1Rb3nkoccuVMvtnsFXPfJjr3B3M Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Domain-Id: None Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Domain-Name: None Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Identity-Status: Confirmed Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Is-Admin-Project: True Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Openstack-Nova-Api-Version: 2.92 Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Domain-Id: default Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Domain-Name: Default Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Project-Name: admin Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Role: member,admin,reader Oct 09 00:49:41.621520 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Roles: member,admin,reader Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant: admin Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-Tenant-Name: admin Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User: admin Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Domain-Id: default Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Domain-Name: Default Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Id: d9c8218a78034540ba9ed09b697588a8 Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: X-User-Name: admin Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: method: > Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: exception: Flavor 451 could not be found. Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: action_args: {'id': '451'} Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:49:41.623024 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.625856 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-00e992c4-5538-4606-b301-2832e77a807d admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105437) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:49:41.627103 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-00e992c4-5538-4606-b301-2832e77a807d admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.396787 Oct 09 00:49:41.627103 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 24/49] 10.222.0.33 () {60 vars in 1193 bytes} [Sun Oct 9 00:49:41 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 398 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 09 00:49:41.633203 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-052aa46c-8602-48d2-8dab-1374dd3a0f0f admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:41.647969 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-052aa46c-8602-48d2-8dab-1374dd3a0f0f admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018283 Oct 09 00:49:41.648510 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 26/50] 10.222.0.33 () {60 vars in 1210 bytes} [Sun Oct 9 00:49:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:49:41.657548 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:41.672566 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018512 Oct 09 00:49:41.673082 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 25/51] 10.222.0.33 () {60 vars in 1210 bytes} [Sun Oct 9 00:49:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 00:49:41.682173 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2f6b5e76-5743-4e77-b0a7-d62b1ddfa8bf admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:41.687519 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.wsgi [None req-2f6b5e76-5743-4e77-b0a7-d62b1ddfa8bf admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2f6b5e76-5743-4e77-b0a7-d62b1ddfa8bf admin admin] Request method failure captured: Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Accept: application/json Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Accept-Encoding: gzip, deflate Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Connection: keep-alive Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Content-Length: 0 Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Host: 10.222.0.33 Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Openstack-Api-Version: compute 2.92 Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Openstack-System-Scope: None Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: User-Agent: python-novaclient Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Auth-Token: gAAAAABjQhqkao7dd8gAqmm5BJHtI0OgMg_UX_KUmwNJtf_oVsrRiJCmNhkcHCd4X91jWZ-fb8sGne2CMgzG60Al7LdC79UIG_JJ-JeIZADLkYE0YVzLBoBFOUBTnMV4Suv_fDZtk54fk8p8Jc8cAq2M1nbJ1Rb3nkoccuVMvtnsFXPfJjr3B3M Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Domain-Id: None Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Domain-Name: None Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Identity-Status: Confirmed Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Is-Admin-Project: True Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Openstack-Nova-Api-Version: 2.92 Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Domain-Id: default Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Domain-Name: Default Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Project-Name: admin Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Role: member,admin,reader Oct 09 00:49:41.689036 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Roles: member,admin,reader Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/34e0fbee16724df3abcac7131a4822aa"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant: admin Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant-Id: 34e0fbee16724df3abcac7131a4822aa Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-Tenant-Name: admin Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User: admin Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Domain-Id: default Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Domain-Name: Default Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Id: d9c8218a78034540ba9ed09b697588a8 Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: X-User-Name: admin Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: method: > Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: exception: Flavor 451 could not be found. Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: action_args: {'id': '451'} Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:49:41.690574 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:41.692490 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: ERROR nova.api.openstack.wsgi Oct 09 00:49:41.693762 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-2f6b5e76-5743-4e77-b0a7-d62b1ddfa8bf admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105436) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:49:41.695001 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-2f6b5e76-5743-4e77-b0a7-d62b1ddfa8bf admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011809 Oct 09 00:49:41.695001 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 27/52] 10.222.0.33 () {60 vars in 1193 bytes} [Sun Oct 9 00:49:41 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 09 00:49:42.904132 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:42.905170 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001448 Oct 09 00:49:42.905555 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 26/53] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:43.425112 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-c553dff7-e5f8-461e-8b65-a4de1b0392be admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:43.444790 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-c553dff7-e5f8-461e-8b65-a4de1b0392be admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.399655 Oct 09 00:49:43.445322 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 28/54] 10.222.0.33 () {64 vars in 1235 bytes} [Sun Oct 9 00:49:43 2022] POST /compute/v2.1/flavors => generated 404 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:49:44.651997 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:44.652991 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Oct 09 00:49:44.653378 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 27/55] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:45.172493 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-f792ab45-562d-4089-9a98-4e86becd0ce0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:45.190795 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-f792ab45-562d-4089-9a98-4e86becd0ce0 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.402057 Oct 09 00:49:45.191337 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 29/56] 10.222.0.33 () {64 vars in 1235 bytes} [Sun Oct 9 00:49:44 2022] POST /compute/v2.1/flavors => generated 405 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:49:46.403953 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:46.404951 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Oct 09 00:49:46.405339 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 28/57] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:46.922739 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-8739d9e7-776a-4c84-a664-1f5c1fe9b5b9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:46.941384 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-8739d9e7-776a-4c84-a664-1f5c1fe9b5b9 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.400317 Oct 09 00:49:46.941922 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 30/58] 10.222.0.33 () {64 vars in 1235 bytes} [Sun Oct 9 00:49:46 2022] POST /compute/v2.1/flavors => generated 407 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:49:48.155213 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:48.156284 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Oct 09 00:49:48.156679 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 29/59] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:48.667662 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-569bf7a7-162a-4a57-bc6a-3fff548b9593 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:49:48.687117 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-569bf7a7-162a-4a57-bc6a-3fff548b9593 admin admin] 10.222.0.33 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.393345 Oct 09 00:49:48.687701 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 31/60] 10.222.0.33 () {64 vars in 1235 bytes} [Sun Oct 9 00:49:48 2022] POST /compute/v2.1/flavors => generated 409 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:49:49.899947 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:49.901525 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002078 Oct 09 00:49:49.901921 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 30/61] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:49:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:49:50.417608 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-8adfba27-2921-4cb0-ab66-68df510464f0 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:49:50.438331 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-8adfba27-2921-4cb0-ab66-68df510464f0 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.399361 Oct 09 00:49:50.438861 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 32/62] 10.222.0.33 () {60 vars in 1202 bytes} [Sun Oct 9 00:49:50 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:50:46.661144 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:50:46.662154 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-8c6be838-6c81-4d3e-b6ff-7c8c1a7c053e admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Oct 09 00:50:46.662559 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 31/63] 10.222.0.33 () {54 vars in 931 bytes} [Sun Oct 9 00:50:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:50:46.673344 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:50:47.050674 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-4f9f9e76-6f06-49fb-9875-a73775d2fab3 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:50:47.071825 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-4f9f9e76-6f06-49fb-9875-a73775d2fab3 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.403098 Oct 09 00:50:47.072368 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 33/64] 10.222.0.33 () {60 vars in 1292 bytes} [Sun Oct 9 00:50:46 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:50:47.093834 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_api:SELECT=14 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:50:47.097550 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-4eeb8d5a-be07-4ab2-b560-d9fb2864353a admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:50:47.104840 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-4eeb8d5a-be07-4ab2-b560-d9fb2864353a admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012433 Oct 09 00:50:47.105355 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 32/65] 10.222.0.33 () {60 vars in 1283 bytes} [Sun Oct 9 00:50:47 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:50:47.116879 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-1464dd1c-4828-4937-854b-da759db5e33e admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:50:47.124641 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-1464dd1c-4828-4937-854b-da759db5e33e admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011325 Oct 09 00:50:47.125160 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 34/66] 10.222.0.33 () {60 vars in 1286 bytes} [Sun Oct 9 00:50:47 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:50:47.136065 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:50:47.143697 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011029 Oct 09 00:50:47.144209 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 33/67] 10.222.0.33 () {60 vars in 1286 bytes} [Sun Oct 9 00:50:47 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:50:47.155010 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-0f5f670c-a47a-4190-8976-cca4005942b1 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:50:47.162047 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-0f5f670c-a47a-4190-8976-cca4005942b1 admin admin] 10.222.0.33 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010477 Oct 09 00:50:47.162560 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 35/68] 10.222.0.33 () {60 vars in 1283 bytes} [Sun Oct 9 00:50:47 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 11 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:53:51.619565 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:53:51.620634 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Oct 09 00:53:51.621036 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 34/69] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:53:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:53:51.758846 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_api:SELECT=6 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:53:51.917068 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:53:51.921769 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:53:51.922231 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:53:51.923182 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:53:51.923909 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:53:51.924347 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:53:51.924751 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:53:51.955922 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-94ab3d44-a8dc-4231-bbe6-07e4c237aa2c admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.198247 Oct 09 00:53:51.956469 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 36/70] 10.222.0.33 () {60 vars in 1193 bytes} [Sun Oct 9 00:53:51 2022] GET /compute/v2.1/os-services => generated 791 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:53:53.029127 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:53:53.030139 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001418 Oct 09 00:53:53.030545 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 35/71] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:53:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:53:53.223242 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:53:53.226678 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:53:53.227157 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:53:53.227628 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:53:53.228340 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:53:53.228783 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:53:53.229198 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:53:53.248191 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-5a1ddfc8-abfe-46b3-9c84-5753cc12f986 admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.076478 Oct 09 00:53:53.248680 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 37/72] 10.222.0.33 () {60 vars in 1193 bytes} [Sun Oct 9 00:53:53 2022] GET /compute/v2.1/os-services => generated 791 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 00:54:04.704406 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:54:04.705496 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Oct 09 00:54:04.705887 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:54:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:54:04.865347 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_cell0:SELECT=4 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:54:04.866057 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_cell1:SELECT=4 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:54:04.866664 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG dbcounter [-] [105436] Writing DB stats nova_api:SELECT=8 {{(pid=105436) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:54:04.914856 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:54:04.918319 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:54:04.918793 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:54:04.919221 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:54:04.919960 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:54:04.920404 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:54:04.920820 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:54:04.945887 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-18bf3718-7879-4777-a0ea-8e796aa1d01d admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.085849 Oct 09 00:54:04.946435 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 38/74] 10.222.0.33 () {60 vars in 1193 bytes} [Sun Oct 9 00:54:04 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:54:06.014272 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG nova.api.openstack.wsgi [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] Calling method '>' {{(pid=105437) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:54:06.015282 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: INFO nova.api.openstack.requestlog [None req-98ed13e0-0e0f-4627-8e6b-5ec709f93bb5 admin admin] 10.222.0.33 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001428 Oct 09 00:54:06.015716 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: [pid: 105437|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1074 bytes} [Sun Oct 9 00:54:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:54:06.206761 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG nova.api.openstack.wsgi [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Calling method '>' {{(pid=105436) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:54:06.210273 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:54:06.210753 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:54:06.211183 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:54:06.212001 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:54:06.212445 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:54:06.212883 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: DEBUG oslo_concurrency.lockutils [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105436) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:54:06.235390 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: INFO nova.api.openstack.requestlog [None req-b331ab20-2b22-4f7e-8c93-3afabdb39fa9 admin admin] 10.222.0.33 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.082477 Oct 09 00:54:06.235939 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: [pid: 105436|app: 0|req: 39/76] 10.222.0.33 () {60 vars in 1193 bytes} [Sun Oct 9 00:54:06 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 00:55:03.801741 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: Sun Oct 9 00:55:03 2022 - graceful shutdown triggered... Oct 09 00:55:03.802277 dv-cinder-master-3-iscsi devstack@n-api.service[105436]: Gracefully killing worker 1 (pid: 105436)... Oct 09 00:55:03.801751 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 09 00:55:03.802806 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: Gracefully killing worker 2 (pid: 105437)... Oct 09 00:55:03.941832 dv-cinder-master-3-iscsi devstack@n-api.service[105437]: DEBUG dbcounter [-] [105437] Writing DB stats nova_api:SELECT=4 {{(pid=105437) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:04.803564 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: worker 1 buried after 1 seconds Oct 09 00:55:04.804004 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: worker 2 buried after 1 seconds Oct 09 00:55:04.804004 dv-cinder-master-3-iscsi devstack@n-api.service[105435]: goodbye to uWSGI. Oct 09 00:55:04.805147 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 09 00:55:04.805681 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 09 00:55:04.808552 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 09 00:55:04.817534 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 9 00:55:04 2022] *** Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: nodename: dv-cinder-master-3-iscsi Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: machine: x86_64 Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: clock source: unix Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: pcre jit disabled Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: detected number of CPU cores: 4 Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: current working directory: / Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: detected binary path: /usr/bin/uwsgi-core Oct 09 00:55:04.819688 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: your processes number limit is 47680 Oct 09 00:55:04.820436 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: your memory page size is 4096 bytes Oct 09 00:55:04.820436 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: detected max file descriptor number: 2048 Oct 09 00:55:04.820436 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: lock engine: pthread robust mutexes Oct 09 00:55:04.820436 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: thunder lock: enabled Oct 09 00:55:04.820436 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 09 00:55:04.820436 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 09 00:55:04.865820 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: Python main interpreter initialized at 0x55a53cda09c0 Oct 09 00:55:04.865820 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: python threads support enabled Oct 09 00:55:04.865820 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: your server socket listen backlog is limited to 100 connections Oct 09 00:55:04.865820 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: your mercy for graceful operations on workers is 90 seconds Oct 09 00:55:04.866307 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: mapped 403077 bytes (393 KB) for 2 cores Oct 09 00:55:04.866307 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: *** Operational MODE: preforking *** Oct 09 00:55:04.866543 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: *** uWSGI is running in multiple interpreter mode *** Oct 09 00:55:04.866543 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: spawned uWSGI master process (pid: 120631) Oct 09 00:55:04.866752 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 09 00:55:04.867431 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: spawned uWSGI worker 1 (pid: 120632, cores: 1) Oct 09 00:55:04.867689 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: spawned uWSGI worker 2 (pid: 120633, cores: 1) Oct 09 00:55:04.867838 dv-cinder-master-3-iscsi devstack@n-api.service[120631]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 09 00:55:05.536297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 09 00:55:05.543746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 09 00:55:06.689449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:55:06.689449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: from cryptography.utils import int_from_bytes Oct 09 00:55:06.690774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:55:06.690774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: from cryptography.utils import int_from_bytes Oct 09 00:55:06.694957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:55:06.694957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: from cryptography.utils import int_from_bytes Oct 09 00:55:06.695406 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 09 00:55:06.695406 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: from cryptography.utils import int_from_bytes Oct 09 00:55:09.152007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 09 00:55:09.152719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 09 00:55:09.152819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 09 00:55:09.153403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 09 00:55:09.153669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 09 00:55:09.153881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 09 00:55:09.154331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.154443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 09 00:55:09.154565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.154663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 09 00:55:09.154751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.154993 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 09 00:55:09.155130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.155300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 09 00:55:09.155419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.155636 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.155740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.155965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156394 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.156971 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.157178 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.157273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.157510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.157617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.157823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.157949 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.158148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.158289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.158446 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.158612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.158756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.158933 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.159195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.159296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.159513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.159635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.159845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.159935 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.160154 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.160373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.160491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.161733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.161733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162589 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.162917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163646 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.163973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164414 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.164944 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.165058 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.165208 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.165323 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.166584 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.166724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.166836 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.166922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167554 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.167688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.168791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.168871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.168956 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169038 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169404 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.169911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170661 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.170925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171351 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171761 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.171958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172188 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172753 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.172996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.173123 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.173309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.173437 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.173623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.173750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.173951 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.174090 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.174255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.174403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.174557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.174714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.174860 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.175026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.175160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.175338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.175461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.175686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.175785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176111 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.176969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.177072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.177268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.177359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.177582 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.177674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.177911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.33 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.33 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178842 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.178943 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.179157 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.179273 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.179464 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.179575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.179806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.179917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.180125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.180224 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.180429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.180528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.180733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.180832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181449 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.181915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.182907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.183121 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.183233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.183423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.183529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.183750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.183860 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184161 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184715 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184827 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.184957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.185976 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.186168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.186269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.186469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.186569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.186773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.186877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.187075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.187177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.187377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.187478 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.187731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.187838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188356 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188667 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.188969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.189133 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.189273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.189377 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.190115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.190222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.190418 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.190520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.190721 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.190821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191040 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191666 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.191982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192729 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.192994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.193186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.193267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.193490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.193651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.193797 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.193885 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.194096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.194184 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.194398 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.194493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.194698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.194808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195106 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.195923 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196602 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.196913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.197135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.197217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.197440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.197522 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.197742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.197832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.198045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.198191 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.198362 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 09 00:55:09.198467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.198696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.198797 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.199023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.199204 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.199361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.199695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.199869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.200965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.201066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.201273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.201374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.201590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.201696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.201902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.202000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.202208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.202312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.202518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.202616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.202931 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203025 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203177 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.203896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.204107 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.204203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.204420 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.204527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.204727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.204828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.205045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.206127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.206318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.206424 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.206512 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.206929 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.207039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.207124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.207198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.207783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.207922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208145 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208430 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.208826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.209363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.209471 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.209597 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.209711 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.209885 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.209982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210427 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210603 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.210963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211047 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211130 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211474 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.211946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.212049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.212291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.212387 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.212614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.212715 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.212953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213254 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213664 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.213967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.214186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.214280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.214497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.214607 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.214806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.214913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.215113 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.215219 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.215411 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.215509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.215736 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.215848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216344 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216452 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.216961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.217055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.217281 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.217379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.217583 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.217684 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.217909 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218830 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.218931 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.219137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.219236 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.219434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.219526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.219768 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.219874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.220079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.220172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.220381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.220481 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.220700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.220793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.221030 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.221132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.221334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.221429 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.221838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.221992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222178 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222862 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.222981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.223171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.223276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.223476 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.223571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.223809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.223915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.224129 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.224231 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.224439 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.224541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.224746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.224857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.225075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.225168 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.225402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.225514 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.225699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.225807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226436 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.226952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.227995 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.228171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.228291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.228478 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.228610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.228779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.228916 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229507 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229678 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.229975 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.230105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.230419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.230597 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.230724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.230899 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.231032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.231347 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232216 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232744 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.232963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.233109 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.233279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.233448 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.233925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234049 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234243 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234559 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234664 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234871 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.234985 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.235184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.235296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.235495 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.235620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.235843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.235972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.236163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.236499 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.236648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.236801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.236955 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.237099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.237240 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.237420 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.237574 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.237724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.237879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238324 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238497 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238817 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.238970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.239147 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.239293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.239498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.239673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.239884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240478 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.240929 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241085 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241515 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.241972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.242121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.242280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.242421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.242572 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.242718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.242884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243177 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.243978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.244122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.244263 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.244404 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.244582 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.244737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.244897 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.245039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.245181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.245334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.245583 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.245734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.245901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246210 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246363 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.246966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.247116 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.247274 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.247417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.247560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.247684 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.247867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248018 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248595 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.248915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249364 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249777 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.249961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.250088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.250942 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.251116 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.251267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.251441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.251621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.251776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.251776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.252601 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.252753 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.252912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253216 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253522 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253822 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.253981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254274 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.254869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.255229 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.255551 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.255884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.256243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.256564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.256876 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.257199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.257525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.257842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.258163 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.258480 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.258795 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.259121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.259440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.259840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.260065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.260222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.260423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.260580 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.260764 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.260920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.261067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.261211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.261394 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.261562 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.261734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.261895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262043 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.262969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.263115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.263269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.263426 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.263722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.263893 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.264043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.264190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.264464 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.264770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.264957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.265114 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.265293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.265458 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.265608 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.265764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.266992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.267141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.267296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.267444 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.267649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.268648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.268816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.268982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.269151 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.269151 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.269447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.269595 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.269953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.270125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.270276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.270423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.270569 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.270716 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.270869 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.271028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.271184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.271332 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.271488 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.271680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.271842 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272003 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272159 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272610 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272610 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.272897 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273044 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273623 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273777 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.273942 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.274096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.274096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.274385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.274558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.274785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.274785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.275203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.275365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.275365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.275686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.275839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.275986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.276156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.276156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.276437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.276583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.276729 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.276874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277224 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277665 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.277976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.278383 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.278546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.278694 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.278839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.278998 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.279156 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.279310 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.279458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.279692 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.279858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280328 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.280924 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281666 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.281979 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.282124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.282288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.282442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.282599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.282754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.282909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.283056 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.283201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.283387 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.283552 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.283731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.283879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284588 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284763 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.284992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.285531 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.285690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.285844 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.285991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.286145 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.286299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.286299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.286599 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.286747 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.286901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287364 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287512 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.287995 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.288150 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.288305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.288470 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.288624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.288778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.288934 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.289101 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.289260 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.289418 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.289567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.289712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.289859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290321 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290776 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.290938 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.291086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.291240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.291395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.291549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.291761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.291989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.292148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.292303 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.292457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.292623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.292779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.292936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.293084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.293238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.293391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.293545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.293700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.293855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294150 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.294914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.295155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.295316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.295316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.295743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.295894 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.295894 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.296183 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.296336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.296483 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.296648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.296795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.296940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.297993 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.298146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.298308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.298456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.298612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.298758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.298935 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.299093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.299253 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.299409 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.299556 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.299725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.299892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.300986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.301141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.301296 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.301445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.301599 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.301747 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.301907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302210 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302357 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302668 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.302984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.303132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.303278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.303432 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.303614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.303784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.303940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.304087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.304241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.304241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.304553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.304702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.304849 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305009 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.305963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.306250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.306250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.306538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.306538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.306830 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.306977 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.307131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.307293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.307449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.307616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.307770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.307917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308220 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308383 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308687 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.308989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.309303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.309457 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.309611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.309767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.309914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.310074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.310233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.310381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.310542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.310711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.310868 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311171 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311481 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311825 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.311981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.312129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.312275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.312428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.312592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.312754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.312754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313482 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.313926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.314073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.314227 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.314375 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.315530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.315771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.316033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.316185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.316333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.316526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.316690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.316852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317472 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.317933 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.318079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.318227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.318387 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.318535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.318682 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.318828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.319024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.319184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.319350 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.319510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.319694 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.319864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320646 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.320974 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.321139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.321287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.321449 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.321603 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.321761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.321919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.322978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.323138 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.323293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.323448 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.323628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.323803 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.323952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.324098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.324244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.324485 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.324658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.324807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.324952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.325098 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.325244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.325505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.325726 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.325925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.326189 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.326343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.326499 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.326657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.326804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.326957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.327104 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.327250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.327396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.327551 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.327728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.327877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.328035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.328200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.328356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.328513 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.328662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.328840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329151 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329299 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329624 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.329930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.330089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.330808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.330961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.331111 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.331257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.331438 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.331683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.331881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.332967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.333122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.333277 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.333424 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.333735 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.333900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334195 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334497 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334805 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.334976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.335135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.335290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.335463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.335749 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.335975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.336126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.336389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.336544 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.336723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.336888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.337039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.337206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.337435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.337596 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.337755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.337913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.338975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.339131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.339297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.339452 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.339659 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.339819 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.339978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.340133 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.340289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.340445 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.340606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.340760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.340916 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341064 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341682 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341836 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.341990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.342137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.342297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.342453 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.342601 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.342760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.342907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.343054 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.343209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.343373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.343527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.343698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.343846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344763 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.344954 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.345117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.345285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.345442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.345856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.346112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.346277 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.346538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.346694 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.346850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347301 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347630 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347806 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.347955 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.348109 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.348271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.348426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.348576 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.348730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.348884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349040 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349495 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.349986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.350141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.350141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.350427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.350581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.350730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.350730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.351026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.351174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.351329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.351329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.351708 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.351865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.352990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.353137 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.353283 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.353429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.353599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.353754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.353911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354058 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354212 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.354968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.355122 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.355284 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.355432 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.355629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.355783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.356102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.356331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.356497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.356749 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.356909 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.357075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.357075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.357377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.357534 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.357688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.357853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358647 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358802 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.358956 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.359110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.359273 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.359436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.359624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.359802 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.359958 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.360124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.360360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.360524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.360681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.360834 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.360987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.361150 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.361306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.361501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.361665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.361827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362327 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362481 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362787 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.362953 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.363117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.363273 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.363427 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.363580 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.363811 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.363990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.364145 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.364145 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.364439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.364600 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.364755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.364918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.365082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.365236 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.365407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.365562 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.365715 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.365888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.366210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.366210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.366606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.366779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.366779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.367192 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.367347 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.367501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.367686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.367855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368180 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368345 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368345 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.368969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.369124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.369277 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.369438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.369594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.369764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.369922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.370084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.370257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.370412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.370573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.370741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.370741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.371049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.371203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.371356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.371510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.371728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.371895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.372051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.372204 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.372368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.372530 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.372699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.372862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373323 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373486 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.373964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.374266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.374447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.374603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.374764 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.374921 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.375074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.375235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.375403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.375558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.375558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.375891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.376045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.376362 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.376614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.376614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.376930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.377719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.377719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378805 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.378973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.379129 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.379129 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.379442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.379612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.379781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.379948 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.380104 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.380266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.380436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.380592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.380745 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.380911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.381067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.381221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.381385 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.381546 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.381710 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.381865 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382507 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382825 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.382979 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.383139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.383304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.383469 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.383695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.383866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384340 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384834 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.384988 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.385142 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.385297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.385472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.385628 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.385791 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.385955 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.386119 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.386431 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.386635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.386635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.387012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.387175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.387338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.387501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.387709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.387974 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.388134 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.388287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.388449 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.388613 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.388767 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.388920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.389080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.389242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.389402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.389566 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.389729 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.389885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390685 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.390992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.391145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.391297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.391297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.391712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.391868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.392033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.392199 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.392368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.392368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.392724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.393396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.393553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.393715 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.393883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.394045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.394208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.394368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.394533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.394696 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.394863 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.395018 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.395249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.395577 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.395916 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.396245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.396573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.396907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.397115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.397277 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.397484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.397671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.397837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.398010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.398253 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.398411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.398580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.398744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.398908 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.399062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.399223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.399385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.399548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.399779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.399947 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.400110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.400272 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.400436 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.400608 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.400771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.400771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.401072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.401226 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.401378 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.401539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.401694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.401857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402806 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.402959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.403361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.403536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.403727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.403890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.404046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.404225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.404381 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.404557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.404714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.404875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405191 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405672 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.405980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.406152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.406341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.406699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.406868 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407105 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407274 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407644 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407817 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.407972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.408302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.408545 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.408728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.408891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.409842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.410132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.410291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.410291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.410615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.410777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.410933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.411085 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.411238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.411410 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.411566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.411744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.411898 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.412063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.412226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.412226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.412535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.412698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.412859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.413960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.414122 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.414283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.414443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.414608 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.414761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.414915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.415077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.415240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.415401 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.415555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.415743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.415898 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.416061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.416222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.416383 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.416562 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.416930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.417124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.417347 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.417506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.417662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.417818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.417979 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.418149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.418312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.418468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.418468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.418884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.418884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.419214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.419385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.419548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.419780 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.419957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.420221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.420389 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.420544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.420698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.420860 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421015 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421355 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.421973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.422137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.422300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.422462 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.422624 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.422778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.422932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.423100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.423255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.423423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.423584 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.423766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120632) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.424353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.425405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.425405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.425405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.425405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.426010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.426010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.426313 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.426546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.426990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.427171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.427436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.427899 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.428214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.428526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.428848 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.429162 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.429486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.429811 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.430123 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.430447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.430772 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.431118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.431430 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.431795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.432121 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.432439 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.432749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.433052 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 09 00:55:09.433293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120633) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 09 00:55:09.486231 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO dbcounter [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Registered counter for database nova_api Oct 09 00:55:09.495463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO dbcounter [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Registered counter for database nova_api Oct 09 00:55:09.507783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_db.sqlalchemy.engines [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120632) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.514366 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writer thread running {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.516394 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_db.sqlalchemy.engines [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120633) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.523025 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writer thread running {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.571104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.context [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8734a8b3-2f61-4ffd-a611-f81fb26e2dd6(cell1) {{(pid=120632) load_cells /opt/stack/nova/nova/context.py:464}} Oct 09 00:55:09.575629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:55:09.575845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:55:09.577429 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:55:09.578175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:55:09.578627 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:55:09.578927 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.context [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),8734a8b3-2f61-4ffd-a611-f81fb26e2dd6(cell1) {{(pid=120633) load_cells /opt/stack/nova/nova/context.py:464}} Oct 09 00:55:09.580724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:55:09.582728 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO dbcounter [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Registered counter for database nova_cell0 Oct 09 00:55:09.582920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:55:09.583399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:55:09.585015 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:55:09.585744 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:55:09.586190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:55:09.586804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO dbcounter [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Registered counter for database nova_cell1 Oct 09 00:55:09.588197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:55:09.590172 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO dbcounter [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Registered counter for database nova_cell0 Oct 09 00:55:09.594447 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_db.sqlalchemy.engines [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120632) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.594609 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO dbcounter [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Registered counter for database nova_cell1 Oct 09 00:55:09.596435 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_db.sqlalchemy.engines [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120632) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.602102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_db.sqlalchemy.engines [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120633) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.602341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writer thread running {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.604657 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_db.sqlalchemy.engines [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120633) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.608162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writer thread running {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.608858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writer thread running {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.749119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writer thread running {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.755980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO dbcounter [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Registered counter for database nova_cell0 Oct 09 00:55:09.761973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO dbcounter [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Registered counter for database nova_cell0 Oct 09 00:55:09.762821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_db.sqlalchemy.engines [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120632) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.766425 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_db.sqlalchemy.engines [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120633) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 09 00:55:09.767798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writer thread running {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:09.771353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writer thread running {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 09 00:55:10.092291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: WARNING keystonemiddleware.auth_token [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:55:10.093338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: WARNING keystonemiddleware.auth_token [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:55:10.100289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: WARNING keystonemiddleware.auth_token [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:55:10.101271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: WARNING keystonemiddleware.auth_token [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:55:10.441931 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: WARNING keystonemiddleware.auth_token [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:55:10.447895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: WARNING keystonemiddleware.auth_token [None req-1c545e89-31a8-4630-bf73-b019b96dd622 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:55:10.449657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: WARNING keystonemiddleware.auth_token [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 09 00:55:10.453175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a53cda09c0 pid: 120633 (default app) Oct 09 00:55:10.456290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: WARNING keystonemiddleware.auth_token [None req-941e075c-ac0d-4da8-9cfc-08368f723508 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 09 00:55:10.461847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a53cda09c0 pid: 120632 (default app) Oct 09 00:55:47.061458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:47.063291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:47.064419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:47.065808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:48.676391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1b0c0c36-b49c-45cd-8f72-491b74a4b11f tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Action: 'create', calling method: .version_select at 0x7f72699d54c0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-994248612"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:55:50.281623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1b0c0c36-b49c-45cd-8f72-491b74a4b11f tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 3.233349 Oct 09 00:55:50.282314 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 1/1] 10.222.0.33 () {60 vars in 1155 bytes} [Sun Oct 9 00:55:47 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 3235 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 09 00:55:50.470224 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c8840cdf-ca42-4f47-8e1a-3c0cf67d7cf4 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f72698a93a0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:50.745639 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-902919d2-f5fa-442b-8e92-275e0542e93b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f7269a640d0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:50.780531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:50.781667 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:50.783038 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:50.784290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:55:50.867187 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c8840cdf-ca42-4f47-8e1a-3c0cf67d7cf4 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/003a4b43-3fdf-4a7e-b75f-7c8c223170fb" status: 200 len: 351 microversion: 2.1 time: 0.401036 Oct 09 00:55:50.868194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 2/2] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:55:50 2022] GET /compute/v2.1/os-volumes/003a4b43-3fdf-4a7e-b75f-7c8c223170fb => generated 351 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:51.217389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-902919d2-f5fa-442b-8e92-275e0542e93b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 200 len: 345 microversion: 2.1 time: 0.921474 Oct 09 00:55:51.217959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 1/3] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:55:50 2022] GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 345 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:51.231275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0a64cc1a-ad4a-46fa-99b9-e3b8868c779d tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Action: 'create', calling method: .version_select at 0x7f72698a9a60>, body: {"snapshot": {"volume_id": "a8b5a564-89e0-4cae-9a05-ac4b2a246ff2", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-163821449"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:55:51.723568 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0a64cc1a-ad4a-46fa-99b9-e3b8868c779d tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.496179 Oct 09 00:55:51.724250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 3/4] 10.222.0.33 () {60 vars in 1162 bytes} [Sun Oct 9 00:55:51 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:51.736606 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1fdd17e7-486c-4550-ac35-930e2ca70e2a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f7269897430>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:51.926384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1fdd17e7-486c-4550-ac35-930e2ca70e2a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 286 microversion: 2.1 time: 0.194686 Oct 09 00:55:51.927621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 2/5] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:51 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 286 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:51.940119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-92a0b27e-7241-4707-a57c-0994e47e39fc tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f7269843c10>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:52.064506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-92a0b27e-7241-4707-a57c-0994e47e39fc tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.129134 Oct 09 00:55:52.065434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 4/6] 10.222.0.33 () {58 vars in 1140 bytes} [Sun Oct 9 00:55:51 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:55:52.078793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fdca9538-230e-407b-9ae9-13cb89052719 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698974c0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:52.153838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fdca9538-230e-407b-9ae9-13cb89052719 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 286 microversion: 2.1 time: 0.080858 Oct 09 00:55:52.155662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 3/7] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:52 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 286 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:52.777075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-619769c3-f684-4624-8bf4-d13caaea615c tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f72699d5ee0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:52.881282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-619769c3-f684-4624-8bf4-d13caaea615c tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/569196e1-6d3a-4bc7-9682-da5e68b771cd" status: 200 len: 351 microversion: 2.1 time: 0.108332 Oct 09 00:55:52.881838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 5/8] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:55:52 2022] GET /compute/v2.1/os-volumes/569196e1-6d3a-4bc7-9682-da5e68b771cd => generated 351 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:53.177791 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b918efbf-21de-4435-b878-ce01df7d10da tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f7269897c10>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:53.233095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b918efbf-21de-4435-b878-ce01df7d10da tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 286 microversion: 2.1 time: 0.067923 Oct 09 00:55:53.237034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 4/9] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:53 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 286 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:54.247946 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3c191bc3-6d77-4620-9a80-3945a71ffc67 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698a99d0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:54.284247 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3c191bc3-6d77-4620-9a80-3945a71ffc67 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 286 microversion: 2.1 time: 0.039981 Oct 09 00:55:54.284809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 6/10] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:54 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 286 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:55.163496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-89658325-ec15-49d8-bbde-3d2d06faa40f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: .version_select at 0x7f72698ae280>, body: {"keypair": {"name": "tempest-keypair-1091593546", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKXQRTkYgFZ4wouV4SGIs0n3Gc/2i0YiMrAcV5tU2LhnJTI4IX6DscJ9wDmNfoeQr01aFx4+A6xrhpYxk5XM7KT8Nj7F453+4WH2xjC7R23K+Y6REjfsoXsRfHnlkzDxrQ=="}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:55:55.251207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-89658325-ec15-49d8-bbde-3d2d06faa40f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for project 40e631c5e320489cac9cea85118c6263. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:55:55.258412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-89658325-ec15-49d8-bbde-3d2d06faa40f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for user 1512c91201f14b75a9be35b89c2097ef and project 40e631c5e320489cac9cea85118c6263. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:55:55.296192 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b408179e-da8c-4f61-ad93-6f624f20cd68 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698a9160>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:55.353658 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b408179e-da8c-4f61-ad93-6f624f20cd68 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 286 microversion: 2.1 time: 0.061008 Oct 09 00:55:55.357268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 7/11] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:55 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 286 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:55.358270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-89658325-ec15-49d8-bbde-3d2d06faa40f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.486870 Oct 09 00:55:55.359455 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 5/12] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 00:55:54 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:56.370564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-305ceec8-0c43-46c2-99f2-bc9d036f0ba8 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698a9a60>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:56.452222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-305ceec8-0c43-46c2-99f2-bc9d036f0ba8 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 287 microversion: 2.1 time: 0.085864 Oct 09 00:55:56.455443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 8/13] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:56 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 287 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:56.465111 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a9cd4f63-304f-41c3-bb7a-5fc3fffd7f8b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698191f0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:56.624600 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a9cd4f63-304f-41c3-bb7a-5fc3fffd7f8b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "DELETE /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 202 len: 0 microversion: 2.1 time: 0.163629 Oct 09 00:55:56.625412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 6/14] 10.222.0.33 () {58 vars in 1254 bytes} [Sun Oct 9 00:55:56 2022] DELETE /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:55:56.635250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cfd07568-2c02-4b8a-a099-3cd18196d5bf tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f726a204820>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:56.744368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7ed4c8f9-ce75-49cb-9646-787c135e0bc2 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f7269819ee0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:56.762250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cfd07568-2c02-4b8a-a099-3cd18196d5bf tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 200 len: 286 microversion: 2.1 time: 0.131717 Oct 09 00:55:56.763189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 9/15] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:56 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 286 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:56.845838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7ed4c8f9-ce75-49cb-9646-787c135e0bc2 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/b7f5c443-7c48-4a97-b463-d4325760c8fc" status: 200 len: 351 microversion: 2.1 time: 0.104652 Oct 09 00:55:56.846380 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 7/16] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:55:56 2022] GET /compute/v2.1/os-volumes/b7f5c443-7c48-4a97-b463-d4325760c8fc => generated 351 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:55:56.859135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-30203e33-911c-4cbf-b3ff-a4d6eaf684e1 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f7269843550>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:57.080370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-30203e33-911c-4cbf-b3ff-a4d6eaf684e1 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.224472 Oct 09 00:55:57.081386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 10/17] 10.222.0.33 () {58 vars in 1134 bytes} [Sun Oct 9 00:55:56 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:55:57.095233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-99ada600-cdc1-4977-b1ee-cf151846d29c tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f72697c45e0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:57.301590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-99ada600-cdc1-4977-b1ee-cf151846d29c tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.209971 Oct 09 00:55:57.304476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 8/18] 10.222.0.33 () {58 vars in 1149 bytes} [Sun Oct 9 00:55:57 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 695 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:57.320087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9df0c952-8420-401f-873f-07f04b087d05 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f7269843af0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:57.507535 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9df0c952-8420-401f-873f-07f04b087d05 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes" status: 200 len: 1036 microversion: 2.1 time: 0.192438 Oct 09 00:55:57.508200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 11/19] 10.222.0.33 () {58 vars in 1134 bytes} [Sun Oct 9 00:55:57 2022] GET /compute/v2.1/os-volumes => generated 1036 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:55:57.522584 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f4cb9d16-3448-4b56-9dd9-98cd20045148 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f72697c4d30>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:57.743612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f4cb9d16-3448-4b56-9dd9-98cd20045148 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.225085 Oct 09 00:55:57.744221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 9/20] 10.222.0.33 () {58 vars in 1167 bytes} [Sun Oct 9 00:55:57 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:57.759300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-640a1de3-b3d7-4acd-8ec2-36a953b5e975 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f72698a9790>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:57.775148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c98d7bfd-773c-4908-8ba0-b7f45eeb6e75 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698199d0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:57.826040 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-c98d7bfd-773c-4908-8ba0-b7f45eeb6e75 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] HTTP exception thrown: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c98d7bfd-773c-4908-8ba0-b7f45eeb6e75 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Request method failure captured: Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b HTTP/1.1 Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwShli6kj25zc7lvW3Ue37edhVcdaFSQAoKcULwiVHO6s2zsmIEeBjQCqt7pTaYPRRbR4t4OU-3nRK8tqb0ssiYk-kADfF2TrEOpMyuHYYuN7o7It3hrfDDifVrkdKBp2ZVjDZ0AUSrIV9EKZavmyhK0HrVtRsfJ9-C5YsvRMUrb8c Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:55:57.831030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 8acc3f6ca4a84de99de98217f7d36d81 Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '28c655e4-ee4d-43fa-acda-dd06e5f5685b'} Oct 09 00:55:57.832355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. (HTTP 404) (Request-ID: req-1104989c-dd0f-490c-818c-4b6c1974dd0a) Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.833207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 00:55:57.834239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 09 00:55:57.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c98d7bfd-773c-4908-8ba0-b7f45eeb6e75 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Returning 404 to user: Snapshot 28c655e4-ee4d-43fa-acda-dd06e5f5685b could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:55:57.836413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c98d7bfd-773c-4908-8ba0-b7f45eeb6e75 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b" status: 404 len: 111 microversion: 2.1 time: 0.061053 Oct 09 00:55:57.839168 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 10/21] 10.222.0.33 () {58 vars in 1251 bytes} [Sun Oct 9 00:55:57 2022] GET /compute/v2.1/os-snapshots/28c655e4-ee4d-43fa-acda-dd06e5f5685b => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 00:55:57.843407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-011094f7-2edd-4ffe-9a33-da815ac508ff tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72698195e0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:58.076700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-640a1de3-b3d7-4acd-8ec2-36a953b5e975 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 695 microversion: 2.1 time: 0.321010 Oct 09 00:55:58.087831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 12/22] 10.222.0.33 () {58 vars in 1170 bytes} [Sun Oct 9 00:55:57 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 695 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:58.092761 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1802c0e3-12f9-46a1-a7b2-7a328b5501c8 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f7269851af0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:58.130253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-011094f7-2edd-4ffe-9a33-da815ac508ff tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 202 len: 0 microversion: 2.1 time: 0.290445 Oct 09 00:55:58.134126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 00:55:57 2022] DELETE /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 0 bytes in 295 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:55:58.139651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4323e68c-d6f5-4b5d-bed7-e1eee395041a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72697af670>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:58.245173 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1802c0e3-12f9-46a1-a7b2-7a328b5501c8 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.156007 Oct 09 00:55:58.245770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4323e68c-d6f5-4b5d-bed7-e1eee395041a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 200 len: 344 microversion: 2.1 time: 0.107021 Oct 09 00:55:58.254509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 12/24] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:55:58 2022] GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 344 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:58.264171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 13/25] 10.222.0.33 () {58 vars in 1155 bytes} [Sun Oct 9 00:55:58 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:55:58.269871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ba6dda72-db96-448a-baf6-ade109a77075 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f7269819e50>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:58.402122 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ba6dda72-db96-448a-baf6-ade109a77075 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.136860 Oct 09 00:55:58.402687 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 13/26] 10.222.0.33 () {58 vars in 1188 bytes} [Sun Oct 9 00:55:58 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:58.416689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1c1fd5e7-8358-45ab-9478-ff07d314ddd2 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] Calling method '.version_select at 0x7f72698a99d0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:58.615055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1c1fd5e7-8358-45ab-9478-ff07d314ddd2 tempest-VolumesTestJSON-2056282625 tempest-VolumesTestJSON-2056282625-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1036 microversion: 2.1 time: 0.202036 Oct 09 00:55:58.628166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1155 bytes} [Sun Oct 9 00:55:58 2022] GET /compute/v2.1/os-volumes/detail => generated 1036 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:55:59.261966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9af4fb98-1f5e-4c0a-b8c3-90748072430c tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72697b1310>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:55:59.406526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9af4fb98-1f5e-4c0a-b8c3-90748072430c tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 200 len: 344 microversion: 2.1 time: 0.149039 Oct 09 00:55:59.407091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 14/28] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:55:59 2022] GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:55:59.936201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-298538182", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "93a9ba3d-37f0-48ce-9786-b421e5404102"}], "security_groups": [{"name": "tempest-securitygroup--716288298"}], "key_name": "tempest-keypair-1091593546", "user_data": "}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:00.364793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] validate_networks() for [('93a9ba3d-37f0-48ce-9786-b421e5404102', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:56:00.422087 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ffcf8ac6-2d4c-4144-a2a3-611e3c2ce6b5 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72697b19d0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:00.607419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-ffcf8ac6-2d4c-4144-a2a3-611e3c2ce6b5 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] HTTP exception thrown: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ffcf8ac6-2d4c-4144-a2a3-611e3c2ce6b5 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Request method failure captured: Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 HTTP/1.1 Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwShli6kj25zc7lvW3Ue37edhVcdaFSQAoKcULwiVHO6s2zsmIEeBjQCqt7pTaYPRRbR4t4OU-3nRK8tqb0ssiYk-kADfF2TrEOpMyuHYYuN7o7It3hrfDDifVrkdKBp2ZVjDZ0AUSrIV9EKZavmyhK0HrVtRsfJ9-C5YsvRMUrb8c Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:56:00.609886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 8acc3f6ca4a84de99de98217f7d36d81 Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'a8b5a564-89e0-4cae-9a05-ac4b2a246ff2'} Oct 09 00:56:00.611210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. (HTTP 404) (Request-ID: req-3884c77b-f87e-4659-a672-c107cc86f816) Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.611987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 00:56:00.613069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 09 00:56:00.614033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ffcf8ac6-2d4c-4144-a2a3-611e3c2ce6b5 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Returning 404 to user: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ffcf8ac6-2d4c-4144-a2a3-611e3c2ce6b5 tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 404 len: 109 microversion: 2.1 time: 0.194660 Oct 09 00:56:00.614898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 15/29] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:56:00 2022] GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 109 bytes in 196 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:56:00.630876 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-df2ded94-7d2b-43a2-b057-d8e9f410a45b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f72697af700>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:00.734108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-df2ded94-7d2b-43a2-b057-d8e9f410a45b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] HTTP exception thrown: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-df2ded94-7d2b-43a2-b057-d8e9f410a45b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Request method failure captured: Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: DELETE /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 HTTP/1.1 Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwShli6kj25zc7lvW3Ue37edhVcdaFSQAoKcULwiVHO6s2zsmIEeBjQCqt7pTaYPRRbR4t4OU-3nRK8tqb0ssiYk-kADfF2TrEOpMyuHYYuN7o7It3hrfDDifVrkdKBp2ZVjDZ0AUSrIV9EKZavmyhK0HrVtRsfJ9-C5YsvRMUrb8c Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:56:00.736214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 8acc3f6ca4a84de99de98217f7d36d81 Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'a8b5a564-89e0-4cae-9a05-ac4b2a246ff2'} Oct 09 00:56:00.737428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. (HTTP 404) (Request-ID: req-8f637951-70dd-4cad-8f63-ea7e0e34411c) Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.738139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 09 00:56:00.739341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 09 00:56:00.740423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.741533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.742244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-df2ded94-7d2b-43a2-b057-d8e9f410a45b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Returning 404 to user: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:56:00.743645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-df2ded94-7d2b-43a2-b057-d8e9f410a45b tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 404 len: 109 microversion: 2.1 time: 0.118548 Oct 09 00:56:00.744415 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 16/30] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 00:56:00 2022] DELETE /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 109 bytes in 120 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 00:56:00.757778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8678eeac-0b88-41ae-92d7-52146e5f9c6a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Calling method '.version_select at 0x7f7269819a60>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:00.907180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-8678eeac-0b88-41ae-92d7-52146e5f9c6a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] HTTP exception thrown: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8678eeac-0b88-41ae-92d7-52146e5f9c6a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Request method failure captured: Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 HTTP/1.1 Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwShli6kj25zc7lvW3Ue37edhVcdaFSQAoKcULwiVHO6s2zsmIEeBjQCqt7pTaYPRRbR4t4OU-3nRK8tqb0ssiYk-kADfF2TrEOpMyuHYYuN7o7It3hrfDDifVrkdKBp2ZVjDZ0AUSrIV9EKZavmyhK0HrVtRsfJ9-C5YsvRMUrb8c Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:56:00.910490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/197f7f9effc24c8eb190027e4d47cda3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 197f7f9effc24c8eb190027e4d47cda3 Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1434876137 Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 8acc3f6ca4a84de99de98217f7d36d81 Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1434876137-project Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'a8b5a564-89e0-4cae-9a05-ac4b2a246ff2'} Oct 09 00:56:00.911603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. (HTTP 404) (Request-ID: req-0894d169-698f-4cc4-9fc2-cc272ce1d35a) Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.912722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 00:56:00.913938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 09 00:56:00.914823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8678eeac-0b88-41ae-92d7-52146e5f9c6a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] Returning 404 to user: Volume a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8678eeac-0b88-41ae-92d7-52146e5f9c6a tempest-VolumesSnapshotsTestJSON-1434876137 tempest-VolumesSnapshotsTestJSON-1434876137-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2" status: 404 len: 109 microversion: 2.1 time: 0.163598 Oct 09 00:56:00.916550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 17/31] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 00:56:00 2022] GET /compute/v2.1/os-volumes/a8b5a564-89e0-4cae-9a05-ac4b2a246ff2 => generated 109 bytes in 165 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 00:56:01.269203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1762417221", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "networks": [{"uuid": "bd0c3622-3100-4974-8f51-c7c8194d1e19"}]}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:01.479547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] validate_networks() for [('bd0c3622-3100-4974-8f51-c7c8194d1e19', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:56:02.625196 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:56:02.626257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:56:02.627097 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:56:02.627910 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:56:02.628561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:56:02.668451 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:02.671696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:02.671696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:02.671696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:02.671696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:02.672294 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:02.777546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for project 40e631c5e320489cac9cea85118c6263. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:56:02.804009 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for user 1512c91201f14b75a9be35b89c2097ef and project 40e631c5e320489cac9cea85118c6263. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:56:02.841928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:56:02.844762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:56:03.101833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-be8e05c0-6385-4db4-98da-fafb43389700 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 3.174563 Oct 09 00:56:03.112400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 15/32] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:55:59 2022] POST /compute/v2.1/servers => generated 401 bytes in 3185 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:56:03.128048 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1a9d30e0-7779-41d5-b057-22dafe41b15f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:03.132009 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1a9d30e0-7779-41d5-b057-22dafe41b15f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:03.478774 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:56:03.479278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:56:03.479709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:56:03.480682 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:56:03.481123 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:56:03.499674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:03.500181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:03.504104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:03.504935 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:03.505415 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:03.509120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:03.577223 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Getting quotas for project 88e7a07d04d74e739b7fa17f73018b64. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:56:03.591562 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Getting quotas for user 15e90d2d2b184f339d1d2aacb77bc82b and project 88e7a07d04d74e739b7fa17f73018b64. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:56:03.653222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:56:03.666077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:56:03.820556 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-60bfa35d-75ef-439f-9fc9-d075b95e86b7 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.834591 Oct 09 00:56:03.821316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 18/33] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:56:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 2841 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:56:03.844083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-26bae8ec-4f0b-41a5-8f7b-f9d0920c54c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:03.844870 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-26bae8ec-4f0b-41a5-8f7b-f9d0920c54c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:03.888216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1a9d30e0-7779-41d5-b057-22dafe41b15f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:03.940415 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1a9d30e0-7779-41d5-b057-22dafe41b15f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1237 microversion: 2.1 time: 0.823192 Oct 09 00:56:03.940989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 16/34] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:03 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1237 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:04.967767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-234efa20-deaa-4a2e-af2d-3267080aa96b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:04.968455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-234efa20-deaa-4a2e-af2d-3267080aa96b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:05.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-26bae8ec-4f0b-41a5-8f7b-f9d0920c54c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:05.235021 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-26bae8ec-4f0b-41a5-8f7b-f9d0920c54c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1215 microversion: 2.1 time: 1.402753 Oct 09 00:56:05.238763 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 19/35] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:03 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1215 bytes in 1407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:05.258678 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-234efa20-deaa-4a2e-af2d-3267080aa96b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:05.270743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-234efa20-deaa-4a2e-af2d-3267080aa96b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1237 microversion: 2.1 time: 0.307356 Oct 09 00:56:05.272538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 17/36] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:04 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1237 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:06.264489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:06.265235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:06.285989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:06.289956 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:06.291724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:06.293422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:06.297372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:06.298881 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:06.310523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:06.311149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:06.336502 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:06.337987 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:06.343115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:06.344688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:06.356438 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:06.357077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:06.729334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:06.747257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1a7a9cbc-5d69-46a5-8f56-ecb8e1e77e68 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1233 microversion: 2.1 time: 0.452124 Oct 09 00:56:06.752725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 18/37] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:06 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1233 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:06.910428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:06.929051 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0ce9cd23-5b1b-4370-8cdd-e834028aa3d1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1211 microversion: 2.1 time: 0.670877 Oct 09 00:56:06.934321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 20/38] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:06 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1211 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:07.779099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:07.780067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:07.797424 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:07.807299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:07.807987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:07.812482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:07.817234 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:07.817746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:07.995646 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e8a8c300-338a-4578-a607-63a8f1b670ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: .version_select at 0x7f7268e85790>, body: {"keypair": {"name": "tempest-keypair-2130470652", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCEA6Xx8C+aLrUmr4vbG+YD1Y4d+jjHWcZyV5WLJOvZYJspAeXIxOxX0gizFbdK4BR5KJar9jebCAnyLJyHT98ipck3n58EvgInp+o/8vPR6oWJnWEHb95+2T7ZRG1FO1w=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:08.071527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-e8a8c300-338a-4578-a607-63a8f1b670ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:56:08.083835 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-e8a8c300-338a-4578-a607-63a8f1b670ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:56:08.154587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e8a8c300-338a-4578-a607-63a8f1b670ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.996788 Oct 09 00:56:08.155815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 19/39] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 00:56:07 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 999 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:08.164674 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:08.165289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:08.174536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:08.175329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:08.176049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:08.182321 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:08.183437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:08.184082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:08.203499 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:08.218082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7932c546-41a1-4058-8d16-f999b18e1b66 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1307 microversion: 2.1 time: 0.442841 Oct 09 00:56:08.222565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 21/40] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:07 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1307 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:08.368028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:08.378159 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e99cb180-47c8-44bc-951c-6a6d82c7f798 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1267 microversion: 2.1 time: 0.220436 Oct 09 00:56:08.396616 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 20/41] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:08 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1267 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:09.248256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:09.248783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:09.259464 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:09.259999 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:09.260444 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:09.261158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:09.268899 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:09.270022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:09.413338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:09.413958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:09.426999 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:09.429449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:09.431282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:09.432423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:09.436266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:09.436942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:09.527963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:09.549126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7dd136c3-3e9d-4e4b-aceb-7acc79958710 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1295 microversion: 2.1 time: 0.304276 Oct 09 00:56:09.552926 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 22/42] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:09 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1295 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:09.881288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-974872298", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:10.118559 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] validate_networks() for [('dd937102-41e7-4629-a161-cb0fb39a0995', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:56:10.267505 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:10.289345 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fcf623e4-fc04-4c5b-b731-9ee5621b45f4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1273 microversion: 2.1 time: 0.882187 Oct 09 00:56:10.295090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 21/43] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:09 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1273 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:10.578226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:10.578829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:10.587393 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:10.587932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:10.588388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:10.595776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:10.599264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:10.602745 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:10.892812 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:10.901031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8cca5874-9374-4af6-91a9-fa8f5694befb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1295 microversion: 2.1 time: 0.326699 Oct 09 00:56:10.901588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 22/44] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:10 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1295 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:11.318255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:11.318868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:11.329249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:11.330333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:11.330803 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:11.340512 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:11.344215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:11.344888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:11.929847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:11.942067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-72b2ee6d-fdc0-4a72-9f73-dcd5aadbe15c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1273 microversion: 2.1 time: 0.627450 Oct 09 00:56:11.951865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 23/45] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:11 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1273 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:11.970770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:11.981869 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:12.020888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:12.021403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:12.021967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:12.028511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:12.029013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:12.037064 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:12.726288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:13.330612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:13.369668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-68e0f309-029d-4d56-84b6-905699ffbefa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 1.415078 Oct 09 00:56:13.375299 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 24/46] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:11 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 1423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:13.392615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:13.393518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:13.400597 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:13.401449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:13.402050 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:13.403471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:13.404732 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:13.407119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:13.552296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:13.707427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:56:13.709589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:56:13.710625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:56:13.712719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:56:13.713831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:56:13.739708 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:13.743333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:13.744634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:13.769593 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:13.772870 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.027s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:13.773427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:13.836285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for project f68941040b3049bd9c62e2fc061d94c1. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:56:13.847335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for user 438446d1e6b3402bb18b547c089391f8 and project f68941040b3049bd9c62e2fc061d94c1. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:56:13.887296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:56:13.899089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:56:13.986954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:14.006761 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bab6169d-c66b-4dc0-8320-7a866ed90aac tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.629793 Oct 09 00:56:14.025014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f8c3dffc-a752-4ce8-a3f1-26fb012b4b01 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.469641 Oct 09 00:56:14.027648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 25/47] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:13 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:14.034768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 23/48] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:56:09 2022] POST /compute/v2.1/servers => generated 383 bytes in 4480 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:56:14.051391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7b88b5de-afe2-41d9-9847-c115d58a4bf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:14.052849 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7b88b5de-afe2-41d9-9847-c115d58a4bf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:14.205084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7b88b5de-afe2-41d9-9847-c115d58a4bf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:14.214351 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7b88b5de-afe2-41d9-9847-c115d58a4bf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1221 microversion: 2.1 time: 0.178751 Oct 09 00:56:14.221187 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 26/49] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:14 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1221 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:14.402135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:14.402761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:14.425438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:14.429527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:14.430089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:14.430849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:14.431324 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:14.435152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:15.041278 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:15.041932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:15.062727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:15.067945 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:15.069822 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:15.071859 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:15.074379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:15.080474 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:15.837150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:15.859877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6bd690d4-df12-4716-84fd-03d97229f4ed tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 1.463760 Oct 09 00:56:15.860774 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 24/50] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:14 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 1467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:15.911840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1404850673", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:16.057893 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] validate_networks() for [('dd937102-41e7-4629-a161-cb0fb39a0995', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:56:16.074329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:16.130432 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1eabeeed-c505-4a62-91a6-4a756e99cfe6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 1.086667 Oct 09 00:56:16.133853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 27/51] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:15 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 1105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:16.890712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:16.894082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:16.926647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:16.939755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:16.948068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:16.950023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:16.955168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:16.958671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:17.908386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:17.918664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2863a7b8-07d9-45f9-847b-40000e442921 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 1.032625 Oct 09 00:56:17.932017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 28/52] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:16 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 1047 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:17.964576 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:17.969276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:18.008818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:18.013103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:18.013597 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:18.019423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:18.037209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:18.039780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:18.719357 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:18.730839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-daad173d-6ee5-4f52-9c8a-fc0c095f5cf2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.794989 Oct 09 00:56:18.731422 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 29/53] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:17 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:18.957424 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:18.958751 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:18.966893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:18.971579 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:18.972489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:18.973444 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:18.975635 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:18.976295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.128606 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:56:19.129643 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:56:19.130126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:56:19.131449 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:56:19.131953 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:56:19.152796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:19.153295 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:19.161692 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.164812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:19.168459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:19.170904 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.229466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for project f68941040b3049bd9c62e2fc061d94c1. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:56:19.238672 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for user 438446d1e6b3402bb18b547c089391f8 and project f68941040b3049bd9c62e2fc061d94c1. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:56:19.251158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:56:19.254481 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:56:19.283352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:19.291134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-00fab94c-f294-4e24-b476-bbcd40cdf70e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.338401 Oct 09 00:56:19.293395 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 30/54] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:18 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:19.301607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1632555524", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ef6b019b-3ff0-42c3-b2bd-354f2c856ccc"}], "security_groups": [{"name": "tempest-securitygroup--527263532"}], "key_name": "tempest-keypair-2130470652", "user_data": "}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:19.362129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d46f66e9-1d13-438b-8f02-6a5c60c7c0d8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.466968 Oct 09 00:56:19.364922 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 25/55] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:56:15 2022] POST /compute/v2.1/servers => generated 383 bytes in 3504 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:56:19.446990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] validate_networks() for [('ef6b019b-3ff0-42c3-b2bd-354f2c856ccc', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:56:19.507686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e59fb8b2-1822-415f-8c7c-f398d24da057", "tag": "afb80013-2258-4db2-b056-f0c814dfdeed"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:19.523280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:19.524048 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:19.524824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.575934 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Creating event network-changed:afb80013-2258-4db2-b056-f0c814dfdeed for instance e59fb8b2-1822-415f-8c7c-f398d24da057 on hv-cinder-maste Oct 09 00:56:19.576864 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120632) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 09 00:56:19.577400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b1103f4e-a4be-4a24-8569-586ef9b9742c req-b662c69a-044e-49cf-9a18-fef3216276a1 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120632) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 09 00:56:19.578622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:19.579304 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:19.581266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.582238 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:19.583029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:19.583688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.594125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.compute.rpcapi [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 09 00:56:19.596162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] Releasing lock "compute-rpcapi-router" {{(pid=120632) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 09 00:56:19.618695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4c73d3b0-747b-44a3-b479-827fbbae2eb1 None None] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.247005 Oct 09 00:56:19.621648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 26/56] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:19.635843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ee5a9c59-4854-453b-bde7-e6bfd559bbf2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:19.637511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ee5a9c59-4854-453b-bde7-e6bfd559bbf2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:19.710788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ee5a9c59-4854-453b-bde7-e6bfd559bbf2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:19.719157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ee5a9c59-4854-453b-bde7-e6bfd559bbf2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1222 microversion: 2.1 time: 0.095575 Oct 09 00:56:19.719936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 27/57] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:19 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1222 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:19.900510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-62a374cf-dce3-447d-8de0-87b64a9e3f76 req-286b48a0-b003-453c-926f-390b2227a04e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3247d2ef-8e30-4cee-b097-1957a0a3de81", "tag": "035fc085-892c-4866-9ff4-6bd3d7193e9b"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:19.927633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-62a374cf-dce3-447d-8de0-87b64a9e3f76 req-286b48a0-b003-453c-926f-390b2227a04e service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:19.928140 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-62a374cf-dce3-447d-8de0-87b64a9e3f76 req-286b48a0-b003-453c-926f-390b2227a04e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:19.928573 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-62a374cf-dce3-447d-8de0-87b64a9e3f76 req-286b48a0-b003-453c-926f-390b2227a04e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:19.961478 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-62a374cf-dce3-447d-8de0-87b64a9e3f76 req-286b48a0-b003-453c-926f-390b2227a04e service nova] Creating event network-changed:035fc085-892c-4866-9ff4-6bd3d7193e9b for instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 on hv-cinder-maste Oct 09 00:56:19.970531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-62a374cf-dce3-447d-8de0-87b64a9e3f76 req-286b48a0-b003-453c-926f-390b2227a04e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.235533 Oct 09 00:56:19.971482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 28/58] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:19.992658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:19.994577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:20.005821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.006622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.007251 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.009213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.009837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.010416 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.062083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:56:20.062524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:56:20.062970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:56:20.063374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:56:20.064361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:56:20.076994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.078167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.078845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.080723 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.081545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.082296 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.112095 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:56:20.121769 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:56:20.152302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:56:20.155892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:56:20.197873 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:20.208420 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1bf3416b-d900-4676-99eb-c9d03e764b30 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.228764 Oct 09 00:56:20.225155 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:19 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:20.229808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-48b448bf-eeb2-4c22-9e79-63cb06f9d3fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.932967 Oct 09 00:56:20.231245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 31/60] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:56:19 2022] POST /compute/v2.1/servers => generated 401 bytes in 935 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:56:20.242242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0f7e3016-b9a0-4a7c-b946-00bf2251f4d0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:20.242853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0f7e3016-b9a0-4a7c-b946-00bf2251f4d0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:20.314758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0f7e3016-b9a0-4a7c-b946-00bf2251f4d0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:20.316588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:20.317201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:20.322557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0f7e3016-b9a0-4a7c-b946-00bf2251f4d0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1242 microversion: 2.1 time: 0.084016 Oct 09 00:56:20.323572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 30/61] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:20 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1242 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:20.333939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.334546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.335238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.336082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.336742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.337360 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.558952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:20.574225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6e6da0fb-21a8-40ee-a358-7b33e584f747 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.260792 Oct 09 00:56:20.575265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 32/62] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:20 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:20.745059 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:20.746065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:20.763026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.763862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.765729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.766670 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:20.768082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:20.769675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:20.867686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:20.890917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4ffaabbb-976e-4799-9966-2c584adc45ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1226 microversion: 2.1 time: 0.151134 Oct 09 00:56:20.893779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:20 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1226 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:21.233969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:21.235219 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:21.242677 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.243273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.243929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.244804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.245843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.246437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.347250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:21.347886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:21.353909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=10 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:21.370969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.371558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.372075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.372861 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.373535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.374112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.443294 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:21.452114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ed70f160-1462-4e28-946c-8298028847e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.222098 Oct 09 00:56:21.481391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:21.504889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 33/64] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:21 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:21.532566 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4ed592cb-5dc8-41f5-ade2-68d2cf21e34d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1246 microversion: 2.1 time: 0.189070 Oct 09 00:56:21.535461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 32/65] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:21 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1246 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:21.601731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:21.602343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:21.609392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.609905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.610622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.611788 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.612269 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.612708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.824889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:21.834515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5a70ba21-379a-4d6f-aa1c-99c81ae1631e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.236906 Oct 09 00:56:21.835170 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 34/66] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:21 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:21.914634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:21.915519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:21.931544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.936591 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.937814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:21.941010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:21.946773 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:21.947413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:22.037705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:22.047222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ba644a04-31a8-488b-8ea2-b882b57f338f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1226 microversion: 2.1 time: 0.136744 Oct 09 00:56:22.048727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 33/67] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:21 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1226 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:22.480575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:22.482933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:22.493198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:22.493896 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:22.494507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:22.500019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:22.500833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:22.501911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:22.558358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:22.560042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:22.579284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:22.580682 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:22.581340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:22.582276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:22.582964 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:22.583659 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:22.690974 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:22.708992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-49292ade-8fb0-43eb-b9d8-b43a8ab820ca tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1246 microversion: 2.1 time: 0.154971 Oct 09 00:56:22.709919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 34/68] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:22 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1246 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:22.734019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:22.745959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fa015ae0-172d-4636-9a62-6a1013bc8440 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.269226 Oct 09 00:56:22.746865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 35/69] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:22 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:22.858129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:22.858737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:22.864833 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:22.865317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:22.865757 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:22.866443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:22.866896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:22.867317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.010728 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:23.021925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7d8d0701-1762-4a53-97ee-63c57871a375 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.166492 Oct 09 00:56:23.022689 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 35/70] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:22 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:23.074299 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:23.074827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:23.081093 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:23.081581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:23.082014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.082708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:23.083163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:23.083633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.157592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:23.160964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:23.165868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b8ee9975-e34c-4f02-bcb0-b1089f08dd1e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1226 microversion: 2.1 time: 0.095539 Oct 09 00:56:23.166483 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 36/71] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:23 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:23.732359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:23.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:23.739631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:23.740120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:23.740557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.741265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:23.741714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:23.742132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.771837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:23.773484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:23.781032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:23.781785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:23.782393 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.784281 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:23.784981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:23.785515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:23.806400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:23.815981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-91dc1109-d30d-487a-9fae-b2db4f53f244 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1246 microversion: 2.1 time: 0.087446 Oct 09 00:56:23.816874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 36/72] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:23 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1246 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:23.928224 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:23.935855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6eb0902c-5126-4601-942e-663d03a1cf4d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.169327 Oct 09 00:56:23.936417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 37/73] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:23 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:24.050544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:24.050812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:24.057256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.057753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.058190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.058887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.059341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.059778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.192082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:24.193029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:24.199194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.200182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.200868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.202645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.203256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.203953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.225526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:24.237683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3bf72c18-48fe-4377-a0da-d594b5855c4e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.192021 Oct 09 00:56:24.238257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 37/74] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:24 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:24.283576 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:24.292799 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-eb751890-9781-46b6-8bd0-8fd30baad669 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1226 microversion: 2.1 time: 0.108069 Oct 09 00:56:24.293359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 38/75] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:24 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1226 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:24.839976 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:24.840590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:24.846622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.847107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.847543 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.848268 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.848725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.849150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.908653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:24.916163 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1ac0676a-1f2e-4864-9f42-ee626660224b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1246 microversion: 2.1 time: 0.079995 Oct 09 00:56:24.916723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 38/76] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:24 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1246 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:24.959522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:24.960169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:24.966232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.966538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.967080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:24.967673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:24.968122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:24.968544 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:25.096411 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:25.104039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-16118852-3c39-4b47-9cc4-192046499af4 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.148784 Oct 09 00:56:25.104593 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 39/77] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:24 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:25.259846 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:25.260367 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:25.266233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:25.266701 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:25.267129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:25.267866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:25.268352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:25.268783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:25.317085 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:25.317687 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:25.320586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:25.324072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:25.324549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:25.324983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:25.325666 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:25.326113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:25.326533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:25.386339 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:25.394212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dc6c7b1c-6b01-4ef1-a8c1-41553cf22b89 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1226 microversion: 2.1 time: 0.081553 Oct 09 00:56:25.394780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 40/78] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:25 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1226 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:25.396114 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:25.403829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a39f016d-f2ba-4734-b789-7303055b71fe tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.147878 Oct 09 00:56:25.404374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 39/79] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:25 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:25.938487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:25.939202 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:25.946260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:25.946739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:25.947169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:25.947877 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:25.948329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:25.948746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.008556 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:26.016080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a986ad36-30ae-4a3b-b627-e4a8a5eb2adc tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1246 microversion: 2.1 time: 0.081280 Oct 09 00:56:26.016630 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 41/80] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:25 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1246 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:26.126563 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:26.127183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:26.134316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:26.134888 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:26.135322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.136046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:26.136491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:26.136910 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.156097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:26.267205 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:26.274682 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4fb9e356-6c31-4fa3-91b9-8899428bb102 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.152025 Oct 09 00:56:26.275240 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 40/81] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:26 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:26.416286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:26.416867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:26.423125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:26.423637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:26.424082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.425430 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:26.425887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:26.426341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.426596 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:26.427208 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:26.433005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:26.433482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:26.434004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.434693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:26.435156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:26.435569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:26.485184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:26.493266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fcd5e030-10f1-47a5-9f15-96272a320a28 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1226 microversion: 2.1 time: 0.080871 Oct 09 00:56:26.493822 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 42/82] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:26 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1226 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:26.560311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:26.567860 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6c4071a8-cc23-49f2-a313-0d48daea8aee tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.145517 Oct 09 00:56:26.568412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 41/83] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:26 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:27.038726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:27.039305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:27.046493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.046967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.047391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.048115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.048561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.048977 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.120922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:27.129379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2124b939-7ee3-4628-8015-09ff9bd5f177 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1246 microversion: 2.1 time: 0.094306 Oct 09 00:56:27.129971 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 43/84] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:27 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1246 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:27.297236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:27.297841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:27.304330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.304815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.305282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.305977 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.306431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.306853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.462669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:27.470136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e5f3c6eb-e1fd-4bf7-a744-abba20025bdd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.176821 Oct 09 00:56:27.470698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 42/85] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:27 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:27.515959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:27.516593 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:27.522517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.522998 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.523433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.524222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.524676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.525098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.589835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:27.590447 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:27.596611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.597100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.597537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.598106 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:27.598590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:27.599054 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:27.599506 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:27.607241 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e14ca07e-2b77-4286-b050-016906d45400 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1218 microversion: 2.1 time: 0.095304 Oct 09 00:56:27.608037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 44/86] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:27 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1218 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:27.757393 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:27.765291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5c8b6cd7-00db-4f97-8ed4-56fe4dc08cb5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.179502 Oct 09 00:56:27.765877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 43/87] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:27 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:28.154268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:28.155102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:28.161687 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.162160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.162663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.163276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.163800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.164203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.226647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:28.234185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d0eda854-0f75-4cea-a7dd-c7f808a51938 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1238 microversion: 2.1 time: 0.083670 Oct 09 00:56:28.234749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 45/88] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:28 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1238 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:28.493300 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:28.493908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:28.499829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.500306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.500744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.501424 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.501880 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.502302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.626321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:28.630021 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:28.630625 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:28.634022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-259a1721-d689-49b5-bfa5-17408e463aa2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.144550 Oct 09 00:56:28.634585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 44/89] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:28 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:28.636342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.636810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.637259 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.637930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.638370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.638784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.699800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:28.707185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-43799aaa-e9a4-4a00-baec-8084063d8e6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1218 microversion: 2.1 time: 0.080890 Oct 09 00:56:28.707773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 46/90] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:28 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1218 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:28.789264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:28.789875 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:28.795844 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.796175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.796608 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.797293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:28.797748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:28.798166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:28.929229 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:28.936793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9788f1fe-fbe7-49bd-9111-6cf0c2e3c9a1 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.151464 Oct 09 00:56:28.937347 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 45/91] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:28 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:29.256612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:29.257270 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:29.263032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.263504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.263968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.264643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.265089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.265505 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.325795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:29.333939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-80c4b017-0f5e-456b-85c6-b1a2644a10e2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1238 microversion: 2.1 time: 0.081184 Oct 09 00:56:29.334486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 47/92] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:29 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1238 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:29.656679 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:29.657280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:29.659653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:29.664985 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.665485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.665919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.666592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.667038 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.667453 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.730068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:29.730669 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:29.736703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.737181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.737612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.738298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.738743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.739160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.793753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:29.799823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:29.801492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-47f4635b-e581-4f6f-9e79-5ce86d484271 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.148797 Oct 09 00:56:29.802239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 46/93] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:29 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:29.807298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-02c44d84-9106-4cd8-a9d7-a773b1bd1c6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1218 microversion: 2.1 time: 0.080987 Oct 09 00:56:29.807822 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 48/94] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:29 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1218 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:29.960256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:29.960874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:29.966698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.967308 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.967642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:29.968334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:29.968790 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:29.969207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.323325 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=28 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:30.355981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:30.356891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:30.358752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:30.359500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=22 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:30.364762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:30.365310 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:30.365976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.367074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:30.367637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:30.368170 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.477175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:30.481091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:30.486662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-96f3002f-2df7-4242-86ac-470ca0565f3a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1238 microversion: 2.1 time: 0.133888 Oct 09 00:56:30.488067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 49/95] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:30 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1238 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:30.493981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b25434dd-bf3e-407e-ae88-ab45d89284da tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.538129 Oct 09 00:56:30.503569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 47/96] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:29 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:30.825434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:30.826228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:30.828872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:30.830022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:30.832903 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:30.833378 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:30.833804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.834489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:30.835094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:30.835356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.836295 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:30.836731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:30.837327 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.838313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:30.838913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:30.839342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:30.906945 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:30.917913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4a12f8e6-8df6-4439-bd6b-84eb95427a1b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1274 microversion: 2.1 time: 0.092975 Oct 09 00:56:30.919707 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 48/97] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:30 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1274 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:31.043944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:31.052139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e76da3ee-d97b-4c1f-8525-607bdcee8bf6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.232138 Oct 09 00:56:31.052712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 50/98] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:30 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:31.139369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-dd577ae5-2bd5-4a58-b7e8-5b800b255ea4 req-f3d4c94d-7d74-447a-b752-11bcde5a6a44 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1b50472-3065-47ec-85b8-f52e36a9374c", "tag": "c41ba939-ec56-4a6f-86b0-047036daa405"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:31.150298 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-dd577ae5-2bd5-4a58-b7e8-5b800b255ea4 req-f3d4c94d-7d74-447a-b752-11bcde5a6a44 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.150866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-dd577ae5-2bd5-4a58-b7e8-5b800b255ea4 req-f3d4c94d-7d74-447a-b752-11bcde5a6a44 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.151182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-dd577ae5-2bd5-4a58-b7e8-5b800b255ea4 req-f3d4c94d-7d74-447a-b752-11bcde5a6a44 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:31.163110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-dd577ae5-2bd5-4a58-b7e8-5b800b255ea4 req-f3d4c94d-7d74-447a-b752-11bcde5a6a44 service nova] Creating event network-changed:c41ba939-ec56-4a6f-86b0-047036daa405 for instance a1b50472-3065-47ec-85b8-f52e36a9374c on hv-cinder-maste Oct 09 00:56:31.168240 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-dd577ae5-2bd5-4a58-b7e8-5b800b255ea4 req-f3d4c94d-7d74-447a-b752-11bcde5a6a44 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032476 Oct 09 00:56:31.169970 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 49/99] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:31.514783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:31.515323 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:31.517418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:31.518113 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:31.524405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.524890 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.525502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:31.526198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.526449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.526818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.527055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.527251 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:31.527490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:31.528208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.528727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.529155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:31.602880 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:31.611675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-001f34cb-80d8-4903-9276-a4dcbec13e39 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1238 microversion: 2.1 time: 0.102248 Oct 09 00:56:31.613120 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 51/100] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:31 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1238 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:31.668972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:31.677180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6b3a43a4-223e-4d62-8722-819150a981a3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.163836 Oct 09 00:56:31.677744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 50/101] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:31 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:31.941280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:31.941874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:31.947748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.948230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.948663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:31.949347 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:31.949797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:31.950216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.010568 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:32.018109 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a1d0badb-a37e-4d76-9ab6-d76821f9b66c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1274 microversion: 2.1 time: 0.081113 Oct 09 00:56:32.018679 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 52/102] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:31 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1274 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:32.073973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:32.074568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:32.081344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:32.081831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:32.082263 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.082973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:32.083421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:32.083866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.211766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:32.219288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-419bd210-e44c-4b9a-b0d7-d40088ce8abd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.149083 Oct 09 00:56:32.219888 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 51/103] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:32 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:32.633932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:32.634531 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:32.640290 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:32.640762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:32.641195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.641861 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:32.642300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:32.642712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.660943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:32.699779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:32.700510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:32.702759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:32.707124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:32.707616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:32.708057 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.708745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:32.709197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:32.709619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:32.710689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3465b585-7f47-41b7-a332-7c18d1b887d3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1238 microversion: 2.1 time: 0.080455 Oct 09 00:56:32.711571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 53/104] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:32 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1238 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:32.833732 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:32.842721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fb191240-fc8f-4490-95c7-05ce7341fbd7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.146860 Oct 09 00:56:32.843288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 52/105] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:32 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:33.041284 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:33.042112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:33.047975 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.048455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.048885 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.050071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.050520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.050939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.121012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:33.143724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-90bc72b2-d61f-4017-853a-c52432d7258f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1274 microversion: 2.1 time: 0.106556 Oct 09 00:56:33.155666 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 54/106] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:33 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1274 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:33.244260 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:33.245819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:33.253250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.254042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.254689 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.255571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.256223 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.256781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.544395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:33.553830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2fa35039-55a8-4fd2-9ba7-bf958b76bc88 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.314755 Oct 09 00:56:33.554314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 53/107] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:33 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:33.732806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:33.733466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:33.739527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.740175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.740687 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.741448 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.741998 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.742487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.834165 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:33.843741 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bd29021c-a0db-4564-bbb9-20ee11207a63 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1294 microversion: 2.1 time: 0.114547 Oct 09 00:56:33.860289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 55/108] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:33 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1294 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:33.866412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:33.867030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:33.872814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.873306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.874045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:33.874761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:33.875200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:33.875648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:34.171464 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:34.172611 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:34.174766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:34.184479 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-976e3a17-1097-4dd8-a903-54c51a093c95 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.321712 Oct 09 00:56:34.187465 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:34.188443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:34.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:34.191057 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 54/109] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:33 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:34.191252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:34.191901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:34.192502 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:34.434386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:34.455076 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a957617f-9c9d-430b-b879-bb0b5d10f138 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1280 microversion: 2.1 time: 0.291682 Oct 09 00:56:34.456598 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 56/110] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:34 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1280 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:34.578708 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:34.579326 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:34.588271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:34.588769 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:34.589214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:34.589951 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:34.590382 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:34.590912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:34.869073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:34.870499 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:34.880674 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:34.884551 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:34.885235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:34.886175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:34.887973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:34.888768 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:35.174801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:35.188167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7cbd8392-1407-42ed-adb6-46046923a860 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1300 microversion: 2.1 time: 0.323147 Oct 09 00:56:35.189118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 57/111] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:34 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1300 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:35.216493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:35.218083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:35.228810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:35.232922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:35.233479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:35.234394 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:35.236317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:35.237090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:35.451011 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:35.469989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9924f80b-cfc6-4173-9ae6-cfe3d4d73476 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.894971 Oct 09 00:56:35.484425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 55/112] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:34 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 909 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:35.498885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:35.499555 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:35.513052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:35.513572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:35.514022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:35.514934 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:35.515883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:35.516477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:35.623110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:35.631347 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-88e98d69-c84b-4f8e-b2de-9da4e8b3a199 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.419132 Oct 09 00:56:35.647114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 58/113] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:35 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:35.732447 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:35.740780 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fa9725b2-6cf3-4df9-93d1-d679e280f777 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.256316 Oct 09 00:56:35.748611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 56/114] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:35 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:36.025359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b86736a3-d8f2-450d-a367-4d2ba4edd923", "tag": "8fe52a88-9c39-4ea8-9d6f-ca693b1da44c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:36.038528 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.039056 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.039478 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.059427 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Creating event network-changed:8fe52a88-9c39-4ea8-9d6f-ca693b1da44c for instance b86736a3-d8f2-450d-a367-4d2ba4edd923 on hv-cinder-maste Oct 09 00:56:36.059970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120633) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 09 00:56:36.060550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fd7c9387-a667-49be-bd49-aa259e16c2f3 req-f1b80a05-e55d-426f-a883-0826890e7815 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120633) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 09 00:56:36.061576 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.062022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.062594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.063440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.063919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.064328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.082515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.compute.rpcapi [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 09 00:56:36.083374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] Releasing lock "compute-rpcapi-router" {{(pid=120633) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 09 00:56:36.099699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0e1312fa-80fb-4212-8113-484ece0c06f8 None None] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077814 Oct 09 00:56:36.103424 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 59/115] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:36.213484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:36.214172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:36.220758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.221220 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.222188 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.223013 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.223462 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.223909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.360696 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:36.365167 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:36.370690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4b319a5a-98be-442b-a0fb-4f9979e63784 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.164318 Oct 09 00:56:36.371260 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 57/116] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:36 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:36.497106 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:36.497789 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:36.503889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.504366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.504794 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.505466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.505913 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.506333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.630497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:36.637915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-03ef1651-4ba8-411d-acd8-409f172bb845 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.144961 Oct 09 00:56:36.638467 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 60/117] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:36 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:36.653991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:36.654624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:36.661432 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.661905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.662331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.663004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.663452 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.663901 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.764796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:36.765575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:36.772042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.772524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.772957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.773640 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:36.774088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:36.774507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:36.796886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:36.804822 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-77370842-362c-440e-a840-a0f096db1b61 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.154497 Oct 09 00:56:36.805405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 58/118] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:36 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:36.922220 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:36.930179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c8c478ff-5507-4256-9188-9910880dede3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.169198 Oct 09 00:56:36.930751 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 61/119] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:36 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:37.393197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:37.394012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:37.400261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.400740 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.401176 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.401862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.402309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.402760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.531918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:37.539614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7b77694b-d58f-4bfc-b3f6-7f20c05f05cf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.150338 Oct 09 00:56:37.540245 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 59/120] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:37 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:37.661522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:37.662125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:37.667992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.668460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.668884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.669566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.669989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.670402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.795919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:37.804182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9153bf46-9303-4145-bf9f-38d7057cfabc tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.147008 Oct 09 00:56:37.804734 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 62/121] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:37 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:37.826787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:37.827412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:37.833453 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.833930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.834362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.835041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.835520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.835993 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.958809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:37.959638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:37.966087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.966266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:37.966946 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.967414 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.968131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:37.968585 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:37.969005 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:37.974001 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-db375acd-efee-4a61-a460-250d8c318ce6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.150978 Oct 09 00:56:37.974561 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 60/122] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:37 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:38.192254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:38.200392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bbf9956d-1b14-4239-a660-53182e2477a0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.247095 Oct 09 00:56:38.201044 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 63/123] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:37 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:38.565290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:38.566242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:38.575915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:38.576882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:38.577649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:38.578602 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:38.579936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:38.580586 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:38.738209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:38.746146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d1f1acd2-d520-4cf6-8dd5-ccb610a10523 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.187314 Oct 09 00:56:38.746835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 61/124] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:38 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:38.826994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:38.827850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:38.835549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:38.844698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:38.844698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:38.844698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:38.844698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:38.846200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:38.898774 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-22e38f04-a083-4d94-bce0-f9109c4f9172 req-31a14c7c-2141-42e2-8e50-3b47ec10ddaf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "acc5e0e3-a218-4f87-b4cc-2efeb32e281e", "tag": "033058db-9b72-4aa7-b12d-bf2294dc0aa3"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:38.910202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-22e38f04-a083-4d94-bce0-f9109c4f9172 req-31a14c7c-2141-42e2-8e50-3b47ec10ddaf service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:38.910673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-22e38f04-a083-4d94-bce0-f9109c4f9172 req-31a14c7c-2141-42e2-8e50-3b47ec10ddaf service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:38.911832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-22e38f04-a083-4d94-bce0-f9109c4f9172 req-31a14c7c-2141-42e2-8e50-3b47ec10ddaf service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:38.924741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-22e38f04-a083-4d94-bce0-f9109c4f9172 req-31a14c7c-2141-42e2-8e50-3b47ec10ddaf service nova] Creating event network-changed:033058db-9b72-4aa7-b12d-bf2294dc0aa3 for instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e on hv-cinder-maste Oct 09 00:56:38.930118 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-22e38f04-a083-4d94-bce0-f9109c4f9172 req-31a14c7c-2141-42e2-8e50-3b47ec10ddaf service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034904 Oct 09 00:56:38.944923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 62/125] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:56:38.997473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:38.998161 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:39.004509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:39.004995 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:39.005426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:39.006110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:39.006788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:39.007271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:39.031228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:39.040847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b1f4f332-551d-4a47-a6f6-ec5305f44427 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.217113 Oct 09 00:56:39.043958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 64/126] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:38 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:39.154782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:39.163573 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a7388637-6be1-4ce6-b499-4d41f4595dcd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.170130 Oct 09 00:56:39.164133 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 63/127] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:38 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:39.223724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:39.224341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:39.230288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:39.230766 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:39.231222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:39.231922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:39.232377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:39.232796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:39.385115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:39.392918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-385a73ff-bf47-4ab6-a899-f33bf2311dc8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.173640 Oct 09 00:56:39.393497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 65/128] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:39 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:39.772601 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:39.773217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:39.779654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:39.780408 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:39.781017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:39.781718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:39.782175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:39.782612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:39.926622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:39.935054 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-859d9aaa-fd14-4d4f-a4fe-a8706e76f672 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.167070 Oct 09 00:56:39.935938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 64/129] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:39 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:40.065801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:40.066406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:40.072712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.073124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.073557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.074246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.074694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.075110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.191965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:40.193292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:40.208097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.210197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.210804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.211804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.212821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.213489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.223307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:40.231435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c6810ad0-a39e-4501-b01b-591cba6afa03 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.169974 Oct 09 00:56:40.232732 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 66/130] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:40 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:40.399317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:40.409603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3d1887e7-be9a-47dd-a7e7-e87ec2d5a717 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.224003 Oct 09 00:56:40.411045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 65/131] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:40 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:40.418207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:40.419215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:40.427570 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.428083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.428668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.429441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.429992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.430510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.590678 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:40.607733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0df0c19c-e4ed-4c30-96ca-43a05af9dded tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.194645 Oct 09 00:56:40.608319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 67/132] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:40 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:40.959886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:40.960585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:40.967098 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.967674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.968174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:40.968920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:40.969371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:40.969793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.105764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:41.114143 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1eaa139e-7722-40c5-8ffe-aa3dcb18fa1e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.158854 Oct 09 00:56:41.114757 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 66/133] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:40 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:41.261164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:41.261763 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:41.270915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:41.271397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:41.271867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.272528 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:41.273131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:41.273547 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.434691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:41.435527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:41.443566 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:41.444427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:41.444996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.448467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:41.448680 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:41.449083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:41.449581 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.460705 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d3791e83-ec4c-4646-8380-e98194667d4c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.203268 Oct 09 00:56:41.461264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 68/134] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:41 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:41.639490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:41.640033 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:41.645981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:41.646452 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:41.647215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.648069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:41.648655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:41.649205 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:41.649320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:41.657700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bbe14355-2acc-4f23-b1a1-f99173177ae6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.227634 Oct 09 00:56:41.658280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 67/135] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:41 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:41.893647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:41.903970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1b5eb500-81ab-4292-93f4-fa0ff15a69d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.268240 Oct 09 00:56:41.904554 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 69/136] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:41 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:42.138214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:42.139000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:42.145149 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.145815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.146246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.147095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.147725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.148297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.281460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:42.289784 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0ca647f9-6666-49ab-a958-35e0fe2bdebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.155283 Oct 09 00:56:42.290499 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 68/137] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:42 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:42.482234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:42.482845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:42.489385 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.489877 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.490315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.491025 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.491476 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.491930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.511046 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:42.642054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:42.650026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e598ffc6-4b9b-4149-8972-f88dd1f767ae tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.171665 Oct 09 00:56:42.650584 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 70/138] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:42 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:42.681014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:42.681693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:42.687523 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.687967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.688396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.689075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.689522 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.689930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.827018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:42.835658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-13c74096-4db8-4eed-99d7-28fc22ecaa3d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.158286 Oct 09 00:56:42.836419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 69/139] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:42 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:42.930147 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:42.932836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:42.943161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.944388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.945048 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:42.946013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:42.946670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:42.947658 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:43.208529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:43.227950 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-65f590d3-1c90-4ae3-bb71-6aecb9f58dd7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.296547 Oct 09 00:56:43.228550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 71/140] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:42 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:43.313817 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:43.314422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:43.320920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:43.321394 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:43.321824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:43.322500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:43.322948 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:43.323394 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:43.458809 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:43.468247 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3a608514-842b-48c7-b186-3223828a16d8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.158197 Oct 09 00:56:43.468800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 70/141] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:43 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:43.673031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:43.673709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:43.679921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:43.680399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:43.680829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:43.681504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:43.681948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:43.682363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:43.818344 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:43.826309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-53799e3e-c9ec-4476-97da-98ae516399c1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.157011 Oct 09 00:56:43.826879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 72/142] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:43 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:43.860155 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:43.861127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:43.867297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:43.867822 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:43.868267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:43.868966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:43.869573 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:43.870153 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:44.034323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:44.048419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-38e65a74-6394-49e8-a8cd-70285a56f722 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.193574 Oct 09 00:56:44.050015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 71/143] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:43 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:44.253509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:44.254610 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:44.263496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:44.264281 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:44.265681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:44.266555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:44.267240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:44.267954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:44.493490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:44.494119 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:44.513337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:44.513838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:44.514279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:44.522731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:44.526850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:44.527918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:44.654163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:44.666643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5a9723ba-b123-4ad1-84f2-d1e32cb65a7e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.417376 Oct 09 00:56:44.676364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 73/144] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:44 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:44.684234 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:44.694399 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-37cabbd7-2c17-4e2e-b8f5-adc9a450f51b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.204576 Oct 09 00:56:44.695706 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 72/145] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:44 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:44.852774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:44.853752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:44.861293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:44.863376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:44.864063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:44.864995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:44.865835 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:44.866490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.072377 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:45.073006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:45.079538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:45.080269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:45.080804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.081564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:45.082019 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:45.082673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.130350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:45.140768 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d80f3888-7215-48fb-816e-54abb0fa744a tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.292095 Oct 09 00:56:45.141940 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 74/146] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:44 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:45.239460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:45.247383 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-60c60113-8b69-41a2-9cb6-27cb6dff25f5 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.178698 Oct 09 00:56:45.248250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 73/147] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:45 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:45.691725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:45.692641 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:45.698713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:45.699418 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:45.699867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.700543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:45.701176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:45.701587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.718874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:45.719819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:45.726302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:45.726863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:45.727513 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.728493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:45.728950 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:45.729372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:45.853333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:45.862861 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bfb03038-55fa-4946-8ea9-84190ba50638 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.175032 Oct 09 00:56:45.863441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 75/148] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:45 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:45.871000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:45.879389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3e4c138e-4540-4d0a-8e19-1ffa811991e7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.165641 Oct 09 00:56:45.879997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 74/149] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:45 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:46.165225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:46.165837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:46.167757 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:46.173144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.173631 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.174060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.174947 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.175395 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.175836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.270225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:46.270831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:46.285017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.285520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.285958 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.286659 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.287285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.287738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.292097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:46.360814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:46.372348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-32b55918-de5d-46fc-820c-d51b1bba19af tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.212081 Oct 09 00:56:46.373165 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 76/150] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:46 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:46.450122 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:46.458396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-80763f0e-55ff-49d7-8aaa-d410bce44817 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.191973 Oct 09 00:56:46.460310 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 75/151] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:46 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:46.885963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:46.886568 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:46.892461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.892925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.893939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.894627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.895077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.895499 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.902491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:46.903157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:46.909343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.909824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.910275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:46.910946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:46.911399 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:46.911844 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:47.024925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:47.032725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-996b773b-f7c1-4392-a6b4-c6181c98d4ac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.150513 Oct 09 00:56:47.033489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 77/152] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:46 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:47.039483 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:47.047163 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-45ef738c-d3ae-4257-a255-db772643cbe7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.148487 Oct 09 00:56:47.047737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 76/153] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:46 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:47.397065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:47.397662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:47.403494 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:47.404002 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:47.404427 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:47.405092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:47.406024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:47.406204 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:47.481489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:47.482091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:47.488691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:47.489172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:47.489604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:47.490284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:47.490736 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:47.491158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:47.533548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:47.542122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-51a94290-bbbe-46ab-a843-30cfa112d381 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.148854 Oct 09 00:56:47.542686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 78/154] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:47 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:47.624092 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:47.631717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fae1d8e6-d93a-4b82-adda-ace03aa28814 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.154112 Oct 09 00:56:47.632272 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 77/155] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:47 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:48.054878 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:48.055487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:48.061728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.062205 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.062634 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.063307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.063792 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.064216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.070434 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:48.071181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:48.077313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.077850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.078233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.079135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.079823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.080439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.194103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:48.201679 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fcf28e08-5037-4a94-a344-29df64d852fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.150733 Oct 09 00:56:48.202244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 79/156] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:48 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:48.212394 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:48.220426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b7d9a7c1-b485-4c19-8b64-34f4440e7034 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.153811 Oct 09 00:56:48.220984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 78/157] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:48 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:48.564628 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:48.565231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:48.570985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.571461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.571924 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.572602 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.573049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.573467 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.653272 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:48.653888 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:48.661082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.661569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.662010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.662704 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:48.663160 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:48.663585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:48.709608 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:48.717796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-26111c0f-fa5e-4abf-b1ec-e03aafe998cd tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.156949 Oct 09 00:56:48.723674 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 80/158] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:48 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:48.793829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:48.801345 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3cd45300-0372-4154-86fb-b73fecae0763 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.151928 Oct 09 00:56:48.801909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 79/159] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:48 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:49.224509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:49.225118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:49.231405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.231968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.232399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.233094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.233540 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.233958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.245235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:49.246200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:49.253735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.254485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.255026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.255896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.256595 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.257127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.384042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:49.394875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e8cb1200-2806-4118-b08e-5e99165f784c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.174287 Oct 09 00:56:49.395756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 81/160] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:49 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:49.407290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:49.416095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cb59b0e5-e684-4ab3-b015-997446877861 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.174661 Oct 09 00:56:49.416677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 80/161] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:49 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:49.748231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:49.748231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:49.748658 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.749148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.749879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.750566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.751025 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.751444 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.824049 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:49.824661 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:49.830586 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.831746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.832181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.832871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:49.833324 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:49.833745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:49.883304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:49.890995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6a92c6ce-cf92-424d-9c9c-2f2b3fc22449 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.153019 Oct 09 00:56:49.891556 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 82/162] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:49 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:49.961859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:49.969658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-aa186058-b1bd-4809-b3d4-b54dd28b86f3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.149362 Oct 09 00:56:49.970258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 81/163] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:49 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:50.417627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:50.418230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:50.424121 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.424601 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.425037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:50.425712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.426160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.426581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:50.438030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:50.438666 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:50.441077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:50.446173 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.446648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.447082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:50.447796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.448248 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.448668 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:50.570268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:50.577974 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ca4328fe-b718-47e9-9225-9ed00b7f4047 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.164145 Oct 09 00:56:50.578537 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 83/164] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:50 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:50.586922 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:50.594579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2991c575-14a3-4e32-8cbd-7cef8ef017a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.160300 Oct 09 00:56:50.595130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 82/165] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:50 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:50.913740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:50.914342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:50.920216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.920678 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.921107 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:50.921783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.922228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.922647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:50.992770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:50.993281 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:50.999036 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:50.999517 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:50.999975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:51.000652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:51.001096 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:51.001510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:51.054235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:51.062498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-40371595-3d5a-4b96-9479-c5721eef2d27 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.152549 Oct 09 00:56:51.063340 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 84/166] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:50 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:51.130164 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:51.137772 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d59b7287-a019-4c5f-9558-e0b6a450f5e4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.148961 Oct 09 00:56:51.138342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 83/167] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:50 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:51.601100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:51.601780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:51.608020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:51.608565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:51.608995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:51.609768 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:51.610214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:51.610633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:51.618006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:51.619611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:51.625554 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:51.626350 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:51.626915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:51.627729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:51.628195 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:51.628618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:51.745047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:51.755326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-87b98cec-9a20-4b67-9480-0324329ac604 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.157883 Oct 09 00:56:51.756130 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 85/168] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:51 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:51.764529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:51.772795 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1d7e3297-f251-4274-83e4-3bbba3c51e86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.158642 Oct 09 00:56:51.773550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 84/169] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:51 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:52.086955 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:52.087468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:52.093453 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.093923 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.094379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.095062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.095541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.096013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.160923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:52.161529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:52.167459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.167978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.168415 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.169101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.169553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.169974 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.229501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:52.237986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-47278334-a105-4cec-abeb-64b98a6b1243 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.155392 Oct 09 00:56:52.238755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 86/170] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:52 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:52.304680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:52.312424 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-27d926b6-fb24-4b2d-8983-43f00d0bdc09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.156039 Oct 09 00:56:52.312993 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 85/171] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:52 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:52.783556 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:52.784243 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:52.791866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.803365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:52.804150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:52.806783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.807457 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.809794 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.810434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.812314 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.813299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.814011 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.815498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.816731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:52.817492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:52.818223 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:52.830372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:53.003830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:53.012115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8de6b0b9-27fe-4a12-b859-61a491fdb4aa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.218648 Oct 09 00:56:53.012739 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 86/172] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:52 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:53.119516 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:53.134530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-50516911-32b3-4b70-aca7-edb5f7541a52 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.357095 Oct 09 00:56:53.137069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 87/173] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:52 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:53.169435 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-a209335b-b1d7-45f1-8945-76802bf5df1f req-5cfb85fc-6503-4fc8-ac80-e111a0c5b405 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e59fb8b2-1822-415f-8c7c-f398d24da057", "name": "network-vif-plugged", "status": "completed", "tag": "afb80013-2258-4db2-b056-f0c814dfdeed"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:53.181002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-a209335b-b1d7-45f1-8945-76802bf5df1f req-5cfb85fc-6503-4fc8-ac80-e111a0c5b405 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:53.181803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-a209335b-b1d7-45f1-8945-76802bf5df1f req-5cfb85fc-6503-4fc8-ac80-e111a0c5b405 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:53.182426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-a209335b-b1d7-45f1-8945-76802bf5df1f req-5cfb85fc-6503-4fc8-ac80-e111a0c5b405 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:53.197748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-a209335b-b1d7-45f1-8945-76802bf5df1f req-5cfb85fc-6503-4fc8-ac80-e111a0c5b405 service nova] Creating event network-vif-plugged:afb80013-2258-4db2-b056-f0c814dfdeed for instance e59fb8b2-1822-415f-8c7c-f398d24da057 on hv-cinder-maste Oct 09 00:56:53.204140 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-a209335b-b1d7-45f1-8945-76802bf5df1f req-5cfb85fc-6503-4fc8-ac80-e111a0c5b405 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.190128 Oct 09 00:56:53.207151 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 87/174] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:53.262929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:53.263538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:53.274965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:53.277251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:53.277741 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:53.278428 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:53.278877 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:53.279294 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:53.339433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:53.344171 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:53.361455 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:53.361956 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:53.362459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:53.366630 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:53.369077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:53.370685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:53.599101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:53.613919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d24f1e49-ecab-44bf-b1de-a338e1d269f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.281544 Oct 09 00:56:53.615197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 88/175] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:53 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:53.644906 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:53.655185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5b942709-4a97-4e15-b6dc-4a658a77f993 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.396410 Oct 09 00:56:53.656580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 88/176] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:53 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:54.035508 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:54.036151 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:54.044261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.044956 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.045592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.046578 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.047220 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.047976 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.158470 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:54.163423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:54.177029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.177747 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.178334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.183428 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.184189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.184760 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.302149 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:54.313447 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e47dc9cc-905e-4cbb-9291-a5bbd808d536 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.282896 Oct 09 00:56:54.328156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 89/177] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:54 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:54.460940 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:54.471118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1c0ae193-e1cc-4a88-acd1-2f9dc8a7c390 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.316622 Oct 09 00:56:54.472545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 89/178] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:54 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:54.639085 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:54.639808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:54.649842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.650356 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.650785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.651477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.653378 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.653815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.685275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:54.685935 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:54.692917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:54.699279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.699823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.700655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.702766 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:54.704949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:54.706844 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:54.879234 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:54.890275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8816e7bb-acd1-4172-af5d-eb31bef835c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.255828 Oct 09 00:56:54.894826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 90/179] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:54 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:54.961389 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:54.969704 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-87ec492c-0d4a-4fe7-879c-79a5bf6eff10 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.289563 Oct 09 00:56:54.970318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 90/180] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:54 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:55.245777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3247d2ef-8e30-4cee-b097-1957a0a3de81", "name": "network-vif-plugged", "status": "completed", "tag": "035fc085-892c-4866-9ff4-6bd3d7193e9b"}, {"server_uuid": "a1b50472-3065-47ec-85b8-f52e36a9374c", "name": "network-vif-plugged", "status": "completed", "tag": "c41ba939-ec56-4a6f-86b0-047036daa405"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:55.263377 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.264376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.265094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.331241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] Creating event network-vif-plugged:035fc085-892c-4866-9ff4-6bd3d7193e9b for instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 on hv-cinder-maste Oct 09 00:56:55.332102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] Creating event network-vif-plugged:c41ba939-ec56-4a6f-86b0-047036daa405 for instance a1b50472-3065-47ec-85b8-f52e36a9374c on hv-cinder-maste Oct 09 00:56:55.350100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:55.351102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:55.356315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-aa0eeacd-eca6-4bde-8266-50aa8e6ad205 req-4579730c-d591-48c3-989a-84e2e6828ab3 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.116090 Oct 09 00:56:55.359278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 91/181] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:55 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:55.369433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.369989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.370431 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.371165 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.371632 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.372065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.506735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:55.507407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:55.531007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.532746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.533204 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.540245 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.541209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.541628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.747282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:55.753456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:55.756573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7d54cc08-49ff-4522-a758-f3ef0f473321 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.410206 Oct 09 00:56:55.759286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 91/182] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:55 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:55.763376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f77c5ffb-f71f-4ecb-9eef-439cf4c6d2f6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.270354 Oct 09 00:56:55.763968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 92/183] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:55 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:55.917515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:55.918118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:55.925916 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.926396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.927062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.927995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:55.928529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:55.929019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:55.993276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:55.994034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:56.000602 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:56.003382 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:56.004915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:56.005638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:56.006101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:56.007610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:56.022626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:56:56.085656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:56.103691 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ba9d1797-2d0d-4520-84ad-ac7b2705f40c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.189742 Oct 09 00:56:56.104633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 92/184] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:55 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:56.277681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:56.288989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a2c57633-2b5e-49b4-86a8-dacc5dea2dde tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1315 microversion: 2.1 time: 0.299971 Oct 09 00:56:56.289622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 93/185] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:55 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1315 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:56.786875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:56.789724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:56.791369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:56.792424 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:56.805026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:56.807019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:56.807916 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:56.808577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:56.809285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:56.810217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:56.810866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:56.812119 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:56.812610 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:56.813290 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:56.813734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:56.817518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:57.003200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:57.014141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f410482f-1213-4568-8d7c-2148e170bca5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.233533 Oct 09 00:56:57.014996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 93/186] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:56 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:57.132423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:57.133571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:57.143857 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:57.144909 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:57.145934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:57.146904 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:57.148149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:57.148870 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:57.230599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:57.241367 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a3f276dd-a5b7-498d-b701-d7177b2cdf35 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.458938 Oct 09 00:56:57.241957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 94/187] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:56 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:57.313175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:57.313763 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:57.319892 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:57.320665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:57.321360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:57.322287 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:57.322947 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:57.323576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:57.490978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:57.501365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0de0618e-5a69-4b31-952f-e19014cba929 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1502 microversion: 2.1 time: 0.191768 Oct 09 00:56:57.502339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 95/188] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:57 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1502 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:57.507120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-57bd509a-f74c-433e-a770-bbfb6baac3ed req-faf2c2a4-7652-4186-8302-73eb762caee8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b86736a3-d8f2-450d-a367-4d2ba4edd923", "name": "network-vif-plugged", "status": "completed", "tag": "8fe52a88-9c39-4ea8-9d6f-ca693b1da44c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:57.517463 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-57bd509a-f74c-433e-a770-bbfb6baac3ed req-faf2c2a4-7652-4186-8302-73eb762caee8 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:57.518986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-57bd509a-f74c-433e-a770-bbfb6baac3ed req-faf2c2a4-7652-4186-8302-73eb762caee8 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:57.519614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-57bd509a-f74c-433e-a770-bbfb6baac3ed req-faf2c2a4-7652-4186-8302-73eb762caee8 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:57.535063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:57.549465 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-57bd509a-f74c-433e-a770-bbfb6baac3ed req-faf2c2a4-7652-4186-8302-73eb762caee8 service nova] Creating event network-vif-plugged:8fe52a88-9c39-4ea8-9d6f-ca693b1da44c for instance b86736a3-d8f2-450d-a367-4d2ba4edd923 on hv-cinder-maste Oct 09 00:56:57.552055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e5c16427-24d7-4458-a158-ce659b35bd55 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1362 microversion: 2.1 time: 0.426738 Oct 09 00:56:57.552774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 94/189] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:57 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1362 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:57.570454 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-57bd509a-f74c-433e-a770-bbfb6baac3ed req-faf2c2a4-7652-4186-8302-73eb762caee8 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066550 Oct 09 00:56:57.571484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 96/190] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:58.039699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:58.041068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:58.056053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.056539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.056964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.057648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.058088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.058533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.230115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:58.237689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a71c50a2-5cb8-4513-b2ef-bde6d8a862c0 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.203773 Oct 09 00:56:58.238241 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 95/191] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:58 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:58.266619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:58.267840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:58.276564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.277161 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.277713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.278679 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.280103 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.281450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.445680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:58.458932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4829f4a6-004b-4898-8f18-33674939ace0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.197315 Oct 09 00:56:58.460366 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 97/192] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:58 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:58.578852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:58.579724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:58.587194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.587704 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.588139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.589929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.590561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.591134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.796586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:58.807382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e4d4736d-4bea-4e09-a541-59ebbb63565e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1549 microversion: 2.1 time: 0.232875 Oct 09 00:56:58.827361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 96/193] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:58 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1549 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:58.835250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:58.836473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:58.844336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.845421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.846036 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.847430 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:58.848576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:58.849131 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:58.981428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-246adb07-6ab4-44d4-889d-772d6bc9cf56 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.151443 Oct 09 00:56:58.982195 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 98/194] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 00:56:58 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-interface => generated 285 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:56:59.106469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b16c2eda-f105-49a9-b519-b1fc83fca724"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:59.131619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:59.158876 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.159496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.161849 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.162646 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.163213 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.163734 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.262352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:59.263418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:59.284092 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.286263 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.287634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.289547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.290241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.292450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.524515 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:59.534225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d613f51d-3fab-4cde-a173-28c878549ed4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.276837 Oct 09 00:56:59.534883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 99/195] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:59 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:56:59.565816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:56:59.566443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:56:59.588472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.589004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.589465 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.590179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.590643 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.591074 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.814921 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:56:59.842831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3313dfcc-41f7-4bf6-afa1-97a3d6a92596 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.291366 Oct 09 00:56:59.845723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 100/196] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:56:59 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:56:59.857525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-ab656078-b993-466d-9503-3049614fbdff req-15122257-0b7d-4922-bff4-ac46fbf76b8b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "acc5e0e3-a218-4f87-b4cc-2efeb32e281e", "name": "network-vif-plugged", "status": "completed", "tag": "033058db-9b72-4aa7-b12d-bf2294dc0aa3"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:56:59.874955 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-ab656078-b993-466d-9503-3049614fbdff req-15122257-0b7d-4922-bff4-ac46fbf76b8b service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:56:59.882099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-ab656078-b993-466d-9503-3049614fbdff req-15122257-0b7d-4922-bff4-ac46fbf76b8b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:56:59.883059 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-ab656078-b993-466d-9503-3049614fbdff req-15122257-0b7d-4922-bff4-ac46fbf76b8b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:56:59.896885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-ab656078-b993-466d-9503-3049614fbdff req-15122257-0b7d-4922-bff4-ac46fbf76b8b service nova] Creating event network-vif-plugged:033058db-9b72-4aa7-b12d-bf2294dc0aa3 for instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e on hv-cinder-maste Oct 09 00:56:59.902975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-ab656078-b993-466d-9503-3049614fbdff req-15122257-0b7d-4922-bff4-ac46fbf76b8b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054117 Oct 09 00:56:59.936186 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 101/197] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:56:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:00.015585 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c870807e-703f-4e96-aaa2-c86ac1ed471c tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "POST /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.915386 Oct 09 00:57:00.019637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 97/198] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 00:56:59 2022] POST /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057/os-volume_attachments => generated 194 bytes in 918 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:00.560889 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:00.562856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:00.576919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:00.578291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:00.581291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:00.582903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:00.585047 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:00.586737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:00.875086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:00.876123 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:00.886767 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:00.890454 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:00.892032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:00.894639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:00.897901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:00.899663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:00.901264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3a654a3b-b1ff-4f40-8ceb-a9f5639fc87a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.342442 Oct 09 00:57:00.901482 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:00.902058 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 102/199] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:00 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:01.298426 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:01.306198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2a47434c-46dc-44b7-a642-6310a6c61525 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.438577 Oct 09 00:57:01.307084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 98/200] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:00 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:01.759544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-26821440-dbe8-475c-b66b-2cfc0b3f214e req-8c8cde93-92b1-48ad-a1c4-7806ef4a786b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3247d2ef-8e30-4cee-b097-1957a0a3de81", "tag": "035fc085-892c-4866-9ff4-6bd3d7193e9b"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:01.773128 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-26821440-dbe8-475c-b66b-2cfc0b3f214e req-8c8cde93-92b1-48ad-a1c4-7806ef4a786b service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:01.773671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-26821440-dbe8-475c-b66b-2cfc0b3f214e req-8c8cde93-92b1-48ad-a1c4-7806ef4a786b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:01.774135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-26821440-dbe8-475c-b66b-2cfc0b3f214e req-8c8cde93-92b1-48ad-a1c4-7806ef4a786b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:01.824848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-26821440-dbe8-475c-b66b-2cfc0b3f214e req-8c8cde93-92b1-48ad-a1c4-7806ef4a786b service nova] Creating event network-changed:035fc085-892c-4866-9ff4-6bd3d7193e9b for instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 on hv-cinder-maste Oct 09 00:57:01.832397 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-26821440-dbe8-475c-b66b-2cfc0b3f214e req-8c8cde93-92b1-48ad-a1c4-7806ef4a786b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076734 Oct 09 00:57:01.841385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 103/201] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:57:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:01.928443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:01.929907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:01.938249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:01.939065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:01.941434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:01.944052 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:01.944739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:01.946866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:02.201736 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:02.212952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bfc28726-6d12-4653-9bac-b2cbf381887a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.287250 Oct 09 00:57:02.237858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 99/202] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:01 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:02.332781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:02.361012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:02.381768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:02.384912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:02.385684 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:02.387807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:02.388431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:02.389155 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:02.568215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:02.576701 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-311e47ca-37e2-4bfa-af93-9fb1d55e91c3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.249679 Oct 09 00:57:02.584749 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 104/203] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:02 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:03.245068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:03.246107 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:03.254217 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:03.254746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:03.255792 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:03.256784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:03.257448 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:03.258196 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:03.437023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:03.445306 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-24fec93e-a6a4-454b-a778-192a227655dd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.204275 Oct 09 00:57:03.457163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 100/204] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:03 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:03.600860 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:03.602191 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:03.611215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:03.612644 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:03.613088 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:03.614005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:03.614612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:03.615193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:03.755891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:03.764973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b190ce5b-1862-4646-95df-b6c921b1646a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1322 microversion: 2.1 time: 0.168635 Oct 09 00:57:03.765601 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 105/205] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:03 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1322 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:04.469449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:04.470059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:04.479377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:04.479886 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:04.480317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:04.481011 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:04.481450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:04.485037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:04.767759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:04.776233 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8894a311-8bb2-4d95-b341-af4b1b7fe26c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.311004 Oct 09 00:57:04.776815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 101/206] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:04 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:04.789525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:04.790382 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:04.797727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:04.798217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:04.798660 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:04.799969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:04.800441 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:04.800866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:04.953393 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:04.962004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f397317f-7dbb-4c3c-8248-44eebc787c92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1515 microversion: 2.1 time: 0.176193 Oct 09 00:57:04.962800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 106/207] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:04 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1515 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:04.985500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:04.988140 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:04.994274 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:04.994761 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:04.995201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:04.995928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:04.996576 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:04.996974 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:05.015282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:05.023050 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lazy-loading 'flavor' on Instance uuid b86736a3-d8f2-450d-a367-4d2ba4edd923 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:57:05.151153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ea217bb0-338e-4cdd-802a-4e03ea87ea99 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923/action" status: 200 len: 32 microversion: 2.1 time: 0.169653 Oct 09 00:57:05.151920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 102/208] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:57:04 2022] POST /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923/action => generated 32 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 00:57:05.162224 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:05.163063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:05.170120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:05.170682 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:05.171178 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:05.171982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:05.172662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:05.173314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:05.334454 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:05.342214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c9761a50-52e9-4d7f-abdb-12760792ce56 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.184509 Oct 09 00:57:05.343093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 107/209] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:05 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:05.801209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:05.802027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:05.808770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:05.809257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:05.809703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:05.810390 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:05.810843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:05.811332 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:05.969926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:05.981393 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fc5e8797-12f5-4897-b433-ac8bce56a833 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1367 microversion: 2.1 time: 0.184701 Oct 09 00:57:05.982246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 103/210] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:05 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1367 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:06.365908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:06.366518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:06.372513 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:06.373213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:06.373652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:06.374354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:06.374967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:06.375539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:06.535141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:06.543334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-64c49786-1a43-4eb0-ac42-066ff4267a12 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.181875 Oct 09 00:57:06.544086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 108/211] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:06 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:07.008249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:07.008858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:07.016508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:07.017405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:07.018211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:07.019943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:07.020733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:07.021284 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:07.233079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:07.242153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ba3e6cda-8b40-4e6b-a809-a99dbe8d35d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1557 microversion: 2.1 time: 0.238919 Oct 09 00:57:07.256455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 104/212] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:07 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1557 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:07.266731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:07.267702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:07.274112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:07.274599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:07.275032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:07.276491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:07.277144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:07.278104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:07.444471 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-13716227-6ae4-4974-97ee-1d56a5662bba tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.181499 Oct 09 00:57:07.445236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 109/213] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 00:57:07 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-interface => generated 284 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:07.567250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:07.567989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:07.581948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:07.584060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:07.585739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:07.588169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:07.589370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:07.589921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:08.129976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:08.141216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-20f3efca-9c59-4cf2-96a9-c2e9891f04a2 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.578147 Oct 09 00:57:08.146732 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 105/214] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:07 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:09.172267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:09.172984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:09.179834 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-970d013e-e4fe-411b-b815-66976f307222 req-1cc80ce6-c4a1-47b6-a0b5-6f586bbb9fd7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "acc5e0e3-a218-4f87-b4cc-2efeb32e281e", "tag": "033058db-9b72-4aa7-b12d-bf2294dc0aa3"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:09.182837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:09.183872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:09.199790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-970d013e-e4fe-411b-b815-66976f307222 req-1cc80ce6-c4a1-47b6-a0b5-6f586bbb9fd7 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:09.200512 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-970d013e-e4fe-411b-b815-66976f307222 req-1cc80ce6-c4a1-47b6-a0b5-6f586bbb9fd7 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:09.201524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-970d013e-e4fe-411b-b815-66976f307222 req-1cc80ce6-c4a1-47b6-a0b5-6f586bbb9fd7 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:09.202069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:09.217398 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:09.217843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-970d013e-e4fe-411b-b815-66976f307222 req-1cc80ce6-c4a1-47b6-a0b5-6f586bbb9fd7 service nova] Creating event network-changed:033058db-9b72-4aa7-b12d-bf2294dc0aa3 for instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e on hv-cinder-maste Oct 09 00:57:09.223678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:09.223815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-970d013e-e4fe-411b-b815-66976f307222 req-1cc80ce6-c4a1-47b6-a0b5-6f586bbb9fd7 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049162 Oct 09 00:57:09.224501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 106/215] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:57:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:57:09.226757 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:09.442107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:09.450592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b3898e1e-4195-4b01-ab90-71f2db307d9b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.284260 Oct 09 00:57:09.451161 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 110/216] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:09 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:10.484075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:10.485968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:10.495951 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:10.496756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:10.497933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:10.498707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:10.499800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:10.500293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:10.681378 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:10.690698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-36767208-7eef-4ab1-aff4-6a1c833d66ed tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.219782 Oct 09 00:57:10.691522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 107/217] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:10 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:11.714347 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:11.715233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:11.722514 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:11.722989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:11.723422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:11.724146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:11.724610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:11.725168 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:11.877058 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:11.887975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8ff00df8-b21c-4e98-b121-f0b56d03ddac tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.174613 Oct 09 00:57:11.887975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 111/218] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:11 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:12.911915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:12.913541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:12.923168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:12.924168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:12.925644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:12.926798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:12.927865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:12.928841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:13.118090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:13.130302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8a0802ee-5531-4bfc-924c-b33dc3fdee2f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.224856 Oct 09 00:57:13.131122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 108/219] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:12 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:14.154908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:14.156107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:14.164466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:14.166014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:14.166752 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:14.168018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:14.168633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:14.169373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:14.191532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:14.341852 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:14.351246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ecb517da-0b6a-4639-b542-6fec085ccb34 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.200109 Oct 09 00:57:14.351855 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 112/220] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:14 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:15.374554 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:15.375156 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:15.381920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:15.382673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:15.383354 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:15.384147 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:15.384793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:15.385406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:15.440000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:15.440605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:15.447965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:15.448455 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:15.449621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:15.451811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:15.453061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:15.453510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:15.496843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 00:57:15.577776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:15.591219 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b8dc8d2c-a364-45c3-ab6d-caa7e45ee502 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.220496 Oct 09 00:57:15.592261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 109/221] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:15 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:15.642081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-45002bab-939a-4ec2-91c6-1c71f63e16e2 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 204 len: 0 microversion: 2.1 time: 0.209429 Oct 09 00:57:15.642669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 113/222] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 00:57:15 2022] DELETE /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 00:57:15.651246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:15.651901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:15.660178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:15.660677 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:15.661256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:15.661993 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:15.662530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:15.663117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:15.864023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:15.875535 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7dd18c54-b99b-42e7-ae28-6167e7adfcc6 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1554 microversion: 2.1 time: 0.227985 Oct 09 00:57:15.888387 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 110/223] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:15 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1554 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:16.620571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:16.621847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:16.629205 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:16.630052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:16.630490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:16.631354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:16.632222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:16.632841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:16.804235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:16.816185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d67724a2-70a4-4e7a-906e-a5eb21f019e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.201988 Oct 09 00:57:16.817351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 114/224] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:16 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:16.904415 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:16.906674 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:16.912769 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:16.913647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:16.914083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:16.914781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:16.915399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:16.916059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:17.131438 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:17.143902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2b6a9858-84bf-46a9-8d2b-3c5beea5d5fe tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1554 microversion: 2.1 time: 0.244791 Oct 09 00:57:17.144643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 111/225] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:16 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1554 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:17.845434 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:17.848261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:17.860066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:17.860724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:17.861748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:17.863042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:17.863751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:17.864402 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:18.058859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:18.077576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2e8f86ab-cc4b-4629-b4ed-2adc85897142 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.237959 Oct 09 00:57:18.078916 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 115/226] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:17 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:18.167695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:18.168305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:18.175072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:18.175671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:18.176188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:18.177066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:18.177513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:18.178112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:18.587668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:18.596984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3d063c06-2be6-42cc-a330-617cc2df6c20 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1554 microversion: 2.1 time: 0.433218 Oct 09 00:57:18.597647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 112/227] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:18 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1554 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:19.100240 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-c91cc3c1-a639-40bf-9dea-2d5faea0130e req-09ad7cac-ce42-458d-9c3b-069e0b74bcd7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e59fb8b2-1822-415f-8c7c-f398d24da057", "name": "network-vif-deleted", "tag": "afb80013-2258-4db2-b056-f0c814dfdeed"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:19.108450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:19.109484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:19.120714 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:19.121209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:19.121648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:19.122348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:19.122797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:19.123216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:19.129771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-c91cc3c1-a639-40bf-9dea-2d5faea0130e req-09ad7cac-ce42-458d-9c3b-069e0b74bcd7 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:19.134007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-c91cc3c1-a639-40bf-9dea-2d5faea0130e req-09ad7cac-ce42-458d-9c3b-069e0b74bcd7 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:19.134621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-c91cc3c1-a639-40bf-9dea-2d5faea0130e req-09ad7cac-ce42-458d-9c3b-069e0b74bcd7 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:19.154919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-c91cc3c1-a639-40bf-9dea-2d5faea0130e req-09ad7cac-ce42-458d-9c3b-069e0b74bcd7 service nova] Creating event network-vif-deleted:afb80013-2258-4db2-b056-f0c814dfdeed for instance e59fb8b2-1822-415f-8c7c-f398d24da057 on hv-cinder-maste Oct 09 00:57:19.179646 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-c91cc3c1-a639-40bf-9dea-2d5faea0130e req-09ad7cac-ce42-458d-9c3b-069e0b74bcd7 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082211 Oct 09 00:57:19.180581 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 116/228] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:57:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:19.331821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:19.340034 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c03bb2f8-78fb-4d7f-8cd5-99e3406cc180 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.235211 Oct 09 00:57:19.341269 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 113/229] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:19 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:57:19.624688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:19.631215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:19.645126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:19.645643 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:19.646032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:19.646713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:19.647367 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:19.647721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:19.728314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:19.738387 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e8c0303b-dab5-4268-99bd-9a8656b77ad1 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1512 microversion: 2.1 time: 0.122015 Oct 09 00:57:19.752215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 117/230] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:19 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1512 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:20.363883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:20.365176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:20.371850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:20.372329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:20.372756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:20.373437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:20.374197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:20.374611 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:20.515559 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:20.524245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-500a3338-9d87-4f7e-a368-f8e7534bf282 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.164362 Oct 09 00:57:20.526029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 114/231] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:20 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:20.762553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:20.763356 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:20.765542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=77,nova_api:UPDATE=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:20.773376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:20.774015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:20.774588 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:20.775275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:20.775980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:20.776604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:20.841732 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:20.852593 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-aaf0ac9a-75dd-4a5f-b9f2-74319b9b449b tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1346 microversion: 2.1 time: 0.094100 Oct 09 00:57:20.853701 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 118/232] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:20 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1346 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:21.560400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:21.569637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:21.576961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:21.577447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:21.577870 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:21.578699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:21.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:21.583539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:21.742117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:21.750564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5a06f2f4-9b67-4b2e-a5f8-4c0cf2a96443 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.204674 Oct 09 00:57:21.751247 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 115/233] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:21 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:21.874674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:21.875430 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:21.882684 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:21.883343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:21.883911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:21.884757 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:21.885313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:21.885981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:21.986421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:21.998034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3c04b27d-e9f0-48c8-bf1e-cdd4ed441985 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1346 microversion: 2.1 time: 0.127630 Oct 09 00:57:21.998610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 119/234] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:21 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1346 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:22.777652 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:22.778815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:22.786461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:22.787126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:22.787572 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:22.788298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:22.788741 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:22.789162 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:22.957589 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:22.968239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-08bea79a-3389-4760-b02a-6861fe9a56dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.195308 Oct 09 00:57:22.968985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 116/235] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:22 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:23.021422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:23.022132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:23.029402 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:23.030037 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:23.030690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:23.036733 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:23.037352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:23.037923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:23.089037 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:23.089744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:23.090179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:23.090816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lazy-loading 'fault' on Instance uuid e59fb8b2-1822-415f-8c7c-f398d24da057 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:57:23.138812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '15e90d2d2b184f339d1d2aacb77bc82b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '88e7a07d04d74e739b7fa17f73018b64', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:23.152671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4a08f342-2346-4cf9-bda3-d6f03818d7b8 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 200 len: 1351 microversion: 2.1 time: 0.135702 Oct 09 00:57:23.153477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 120/236] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:23 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 1351 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:23.992929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:23.993468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:24.000129 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:24.000606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:24.001039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:24.001722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:24.002168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:24.002587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:24.131250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:24.134721 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:24.139754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f61ec763-daba-4df3-b1fb-f8f5411a5018 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.151577 Oct 09 00:57:24.140315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 117/237] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:23 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:24.176884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:24.177486 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:24.183422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:24.183941 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:24.184373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:24.185052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:24.185504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:24.185927 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:24.198971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] HTTP exception thrown: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Request method failure captured: Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 HTTP/1.1 Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwgu7bOrzCNSU1Ov1F9wT_8-bhYQF4kY302GX1Po6Y8JbgcvmpT2O3KMYKTCcoO3MqKcXX0B5dlOoNFXnswE0CAaDgaVFM6dH-W_Nyy23pcLIrerYQfhih5RSNgNqyO6z-ZG7jhM-MA_6d6Lg7TkH2icdLj2NoGtipWvJxH56on4iQ Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:57:24.203450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 15e90d2d2b184f339d1d2aacb77bc82b Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.204887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'e59fb8b2-1822-415f-8c7c-f398d24da057'} Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:57:24.205759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.207249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.208600 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Returning 404 to user: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:57:24.208600 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bb4ded95-0866-4349-a6d6-25d14693a9c0 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 404 len: 111 microversion: 2.1 time: 0.032357 Oct 09 00:57:24.208600 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 121/238] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:24 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:57:24.318221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:24.320447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:24.326831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:24.327309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:24.327774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:24.328451 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:24.328898 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:24.329321 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:24.342181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] HTTP exception thrown: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Request method failure captured: Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057/os-volume_attachments HTTP/1.1 Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwgu7bOrzCNSU1Ov1F9wT_8-bhYQF4kY302GX1Po6Y8JbgcvmpT2O3KMYKTCcoO3MqKcXX0B5dlOoNFXnswE0CAaDgaVFM6dH-W_Nyy23pcLIrerYQfhih5RSNgNqyO6z-ZG7jhM-MA_6d6Lg7TkH2icdLj2NoGtipWvJxH56on4iQ Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 00:57:24.347629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 15e90d2d2b184f339d1d2aacb77bc82b Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.349003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'server_id': 'e59fb8b2-1822-415f-8c7c-f398d24da057'} Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:57:24.349676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:24.350949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:24.352824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Returning 404 to user: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:57:24.352824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5910fecf-222f-4355-a918-989937f57dce tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.035482 Oct 09 00:57:24.352824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 118/239] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 00:57:24 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057/os-volume_attachments => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:57:25.163064 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:25.163690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:25.169618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:25.170084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:25.170510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:25.171177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:25.171643 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:25.172060 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:25.297213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:25.304797 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a0f33e14-6cbc-4b10-bb16-344fb85386bb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.146149 Oct 09 00:57:25.305345 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 122/240] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:25 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:26.327780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:26.328307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:26.330623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=57 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:26.336529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:26.337000 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:26.337426 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:26.338099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:26.338543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:26.338959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:26.467645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:26.475683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-acca6f56-75d7-4bb1-b9e8-78308aa5dc09 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.151891 Oct 09 00:57:26.476245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 119/241] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:26 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:27.499555 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:27.500392 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:27.507273 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:27.507783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:27.508215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:27.508893 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:27.509337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:27.509796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:27.661932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:27.662519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:27.663005 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:27.669880 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:27.670373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:27.670807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:27.670908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-529dfb29-79a9-4644-bb5f-7d9b6da9251c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.175757 Oct 09 00:57:27.671523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:27.671698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 123/242] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:27 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:27.672002 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:27.672842 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:27.688962 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] HTTP exception thrown: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Request method failure captured: Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 HTTP/1.1 Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwgu7bOrzCNSU1Ov1F9wT_8-bhYQF4kY302GX1Po6Y8JbgcvmpT2O3KMYKTCcoO3MqKcXX0B5dlOoNFXnswE0CAaDgaVFM6dH-W_Nyy23pcLIrerYQfhih5RSNgNqyO6z-ZG7jhM-MA_6d6Lg7TkH2icdLj2NoGtipWvJxH56on4iQ Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 00:57:27.691698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 15e90d2d2b184f339d1d2aacb77bc82b Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.693142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'e59fb8b2-1822-415f-8c7c-f398d24da057'} Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:57:27.693939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.695239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.696517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Returning 404 to user: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:57:27.696517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-89c81f41-a43c-4355-962d-28888804c47d tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 404 len: 111 microversion: 2.1 time: 0.034559 Oct 09 00:57:27.696517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 120/243] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 00:57:27 2022] DELETE /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 00:57:27.701832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:27.702434 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] [instance: e59fb8b2-1822-415f-8c7c-f398d24da057] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:27.708532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:27.709029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:27.709451 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:27.710239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:27.710735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:27.711182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:27.724684 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] HTTP exception thrown: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Request method failure captured: Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 HTTP/1.1 Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwgu7bOrzCNSU1Ov1F9wT_8-bhYQF4kY302GX1Po6Y8JbgcvmpT2O3KMYKTCcoO3MqKcXX0B5dlOoNFXnswE0CAaDgaVFM6dH-W_Nyy23pcLIrerYQfhih5RSNgNqyO6z-ZG7jhM-MA_6d6Lg7TkH2icdLj2NoGtipWvJxH56on4iQ Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:57:27.726199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/88e7a07d04d74e739b7fa17f73018b64"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 88e7a07d04d74e739b7fa17f73018b64 Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-DeleteServersTestJSON-718467185 Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 15e90d2d2b184f339d1d2aacb77bc82b Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-DeleteServersTestJSON-718467185-project Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.727575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'e59fb8b2-1822-415f-8c7c-f398d24da057'} Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:57:27.731620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. Oct 09 00:57:27.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:57:27.734315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] Returning 404 to user: Instance e59fb8b2-1822-415f-8c7c-f398d24da057 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:57:27.734315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-79a6d186-3dd5-4beb-8727-c2de4b64ad54 tempest-DeleteServersTestJSON-718467185 tempest-DeleteServersTestJSON-718467185-project] 10.222.0.33 "GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057" status: 404 len: 111 microversion: 2.1 time: 0.029532 Oct 09 00:57:27.734315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 124/244] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:27 2022] GET /compute/v2.1/servers/e59fb8b2-1822-415f-8c7c-f398d24da057 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:57:28.695365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:28.696015 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:28.703811 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:28.704391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:28.704892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:28.705654 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:28.706338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:28.707477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:28.842631 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:28.850790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5d5e9934-c58e-46cf-a15b-fa119a33a61f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.160249 Oct 09 00:57:28.851349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 121/245] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:28 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:57:29.874442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:29.875279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:29.883490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:29.884882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:29.886722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:29.888321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:29.889029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:29.890811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:30.119130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:30.140527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f521a425-31b5-4295-a85a-f25f9bdb6e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.270591 Oct 09 00:57:30.145322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 125/246] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:29 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:31.168659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:31.169941 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:31.179020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:31.179636 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:31.180014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:31.180695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:31.181178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:31.185907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:31.398722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:31.408025 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f8eb56d4-447b-4760-b448-afb8cfc6cb08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.243105 Oct 09 00:57:31.409138 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 122/247] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:31 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:32.435676 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:32.436395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:32.443047 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:32.443637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:32.444140 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:32.445137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:32.445572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:32.446181 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:32.686799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:32.704254 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-241ec222-9416-4d93-9231-384827b0356c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.272007 Oct 09 00:57:32.719623 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 126/248] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:32 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:33.730123 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:33.730734 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:33.741643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:33.742098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:33.748573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:33.753026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:33.753566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:33.754894 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:34.264400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:34.296823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-075d5177-ce86-483a-a026-7c733439a229 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.570662 Oct 09 00:57:34.297440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 123/249] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:33 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:35.322700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:35.323479 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:35.331189 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:35.332736 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:35.333872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:35.334890 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:35.335533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:35.336384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:35.546968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:35.560529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c0fd0418-c035-43c5-9f9c-0ec6c8d2701e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.242613 Oct 09 00:57:35.561775 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 127/250] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:35 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:36.585443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:36.586576 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:36.597361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:36.598091 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:36.598721 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:36.603942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:36.604807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:36.605656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:36.952186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:36.963101 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-80c2f732-0ca5-47bd-a72d-c35f3e26de5b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.379814 Oct 09 00:57:36.979254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 124/251] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:36 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:37.986952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:37.988946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:38.011971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:38.012642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:38.013524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:38.018472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:38.020537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:38.022250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:38.272537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:38.286281 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b7dcc2da-0f5c-48d8-8f77-ab3fa91c397e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.304031 Oct 09 00:57:38.298309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 128/252] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:37 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:39.312083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:39.313786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:39.326130 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:39.326543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:39.327132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:39.327831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:39.328452 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:39.328872 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:39.550256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:39.559216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-38f4bba2-d0ec-4916-b41d-b9a86445a1d3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.252716 Oct 09 00:57:39.559967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 125/253] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:39 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:40.584185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:40.584810 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:40.592228 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:40.592866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:40.593883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:40.595198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:40.595934 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:40.596500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:40.638290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:41.420062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:41.433131 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-55e1a625-5853-4964-a0bb-802bee1e796a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.854252 Oct 09 00:57:41.435994 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 129/254] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:40 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 858 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:42.472736 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:42.473368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:42.491661 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:42.496874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:42.497382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:42.498669 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:42.499143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:42.499578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:42.898889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:42.908546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-23080c79-82b6-43b8-b7b4-fe87842d4816 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.443549 Oct 09 00:57:42.925049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 126/255] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:42 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:43.933899 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:43.934470 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:43.944130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:43.945771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:43.946410 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:43.947568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:43.948472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:43.949104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:44.169505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:44.179865 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c6ef3bfd-4f19-4549-985d-569c8125b1ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.251003 Oct 09 00:57:44.180685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 130/256] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:43 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:44.946769 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-89cd3bd3-49be-4956-8aa2-e453918d4817 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Action: 'create', calling method: .version_select at 0x7f7268bba820>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1094622065", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKTGlruoxLILy97utg5XE7C8ju8YqhCyYtBy/uszTqj++IIMK1uXCPrl6mjRVXklzwk0Hgm5kgV5Ka1pl1LprElM6NzfJDCPj19rLS1K54DL62K7j3ksDapo3SgGGJBXmg=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:44.969504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-89cd3bd3-49be-4956-8aa2-e453918d4817 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:57:44.976548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-89cd3bd3-49be-4956-8aa2-e453918d4817 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for user 04bc22d558c2466ba43a456d08bdd6bd and project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:57:45.000954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-89cd3bd3-49be-4956-8aa2-e453918d4817 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.281010 Oct 09 00:57:45.001918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 127/257] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 00:57:44 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:57:45.203391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:45.205529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:45.212839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:45.213494 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:45.214024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:45.214968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:45.215526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:45.216222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:45.430851 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:45.442387 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-714276ed-a6bf-46d4-91f2-ad3032476cde tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.241843 Oct 09 00:57:45.443665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 131/258] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:45 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:46.467616 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:46.468406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:46.480354 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:46.481064 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:46.481663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:46.482511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:46.483108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:46.483700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:46.678287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:46.700920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-82479640-1b2d-47d9-ba56-7f1a97dd1cf3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1521 microversion: 2.1 time: 0.237792 Oct 09 00:57:46.702289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 128/259] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:46 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1521 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:47.731376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:47.732024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:47.739431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:47.740120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:47.748072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:47.748072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:47.748072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:47.748072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:47.934788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:47.944284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fbaedfc8-2ddf-4653-9921-9d8de67c1850 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1501 microversion: 2.1 time: 0.216539 Oct 09 00:57:47.945071 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 132/260] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:47 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1501 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:47.966651 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:47.967382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:47.973973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:47.974453 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:47.975080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:47.975806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:47.976447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:47.977060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:48.159384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:48.169204 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-200380f3-5588-4bc3-845a-dfe9a3f13c7d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1514 microversion: 2.1 time: 0.206233 Oct 09 00:57:48.178628 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 129/261] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:47 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1514 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:48.989531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:48.990020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:49.005211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:49.005906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:49.006317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:49.007322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:49.007859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:49.008268 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:49.058658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5ef9cb6d-2081-41f6-9aad-d8b6a55d334d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/ips" status: 200 len: 146 microversion: 2.1 time: 0.075382 Oct 09 00:57:49.059242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 133/262] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 00:57:48 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/ips => generated 146 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:49.959108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:49.963203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:49.979226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:49.979754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:49.980195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:49.980905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:49.981352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:49.981980 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:50.013442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lazy-loading 'flavor' on Instance uuid a1b50472-3065-47ec-85b8-f52e36a9374c {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:57:50.126338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-062d132d-e7ec-41eb-aa5b-f7d347280bfe tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/action" status: 200 len: 32 microversion: 2.1 time: 0.172137 Oct 09 00:57:50.127262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 130/263] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:57:49 2022] POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/action => generated 32 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 00:57:50.145089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:50.145089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:50.162253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:50.164390 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:50.165303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:50.166100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:50.166639 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:50.167144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:50.367008 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:50.377450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-625293db-66c4-4288-9011-58a6713c1d80 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.240005 Oct 09 00:57:50.380502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 134/264] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:50 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:51.409332 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:51.410195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:51.417514 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:51.418131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:51.418619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:51.420692 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:51.421511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:51.423703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:51.617836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:51.628139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b41e5b8e-05e8-4906-819d-aa25d2b6577d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.225321 Oct 09 00:57:51.629063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 131/265] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:51 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:52.651413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:52.652045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:52.654200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=36 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:52.660369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:52.660851 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:52.661284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:52.661969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:52.662422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:52.662848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:52.796687 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:52.807850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bd5d778d-2ebf-48aa-b474-f87bba1cda6b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.158286 Oct 09 00:57:52.807850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 135/266] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:52 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:53.828359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:53.828960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:53.835240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:53.835749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:53.836179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:53.836858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:53.837305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:53.837731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:53.971038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:53.978838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-83e0ea94-fd7d-40fa-a031-3a20fccec4dd tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.154356 Oct 09 00:57:53.979392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 132/267] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:53 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:55.002329 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:55.002960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:55.009642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:55.010128 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:55.010563 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:55.011278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:55.011762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:55.012187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:55.157246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:55.165771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b9b6a293-2b0b-4e35-b8fc-d5edf13a2510 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.167328 Oct 09 00:57:55.166335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 136/268] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:54 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:55.795537 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "adee1caf-ae25-496d-9f4c-b58d602ff8de"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:57:55.802503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:55.802503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=4 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:55.821697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:55.821697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:55.821697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:55.821697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:55.822391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:55.822391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:55.827682 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:56.135683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a90b4f90-7cc4-4fc3-a1a5-ff4cbcb48577 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.343724 Oct 09 00:57:56.136250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 133/269] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 00:57:55 2022] POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-volume_attachments => generated 194 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:57:56.190640 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:56.191374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:56.200483 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:56.201351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:56.201993 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:56.202912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:56.203365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:56.204006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:56.374459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:56.383309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0e111aea-fb01-4fb1-b181-b6b86f00746c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.198047 Oct 09 00:57:56.383913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 137/270] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:56 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:57.416084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:57.417247 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:57.426605 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:57:57.431748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:57.431748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:57.431748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:57.431748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:57.432070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:57.440162 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:57.634384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:57.641933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-15fba9cf-7fb0-4cb6-bac0-5e286bb68a3f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.232562 Oct 09 00:57:57.642716 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 134/271] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:57 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:58.668341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:58.668962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:58.676095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:58.676587 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:58.677021 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:58.678026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:58.678615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:58.679331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:58.875654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:57:58.884318 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bd400afa-e68d-4776-a0eb-d2a038408026 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.219761 Oct 09 00:57:58.884881 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 138/272] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:58 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:57:59.908497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:57:59.909105 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:57:59.916316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:59.917164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:59.917871 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:57:59.918827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:57:59.919526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:57:59.920126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:00.107705 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:00.118749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-055b0fb0-030c-4b97-ae89-8f666a8adf74 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.214115 Oct 09 00:58:00.119507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 135/273] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:57:59 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:01.142191 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:01.144028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:01.150826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:01.151278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:01.151743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:01.152506 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:01.152960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:01.153433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:01.295485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:01.303821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-eef85dae-8419-4257-806e-38826692e8db tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.165338 Oct 09 00:58:01.304403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 139/274] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:01 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:02.326505 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:02.327111 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:02.333543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:02.334088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:02.334529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:02.335229 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:02.335729 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:02.336168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:02.563318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:02.571161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d3e977bf-18b1-436f-990f-69c42bc4f925 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.248463 Oct 09 00:58:02.571862 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 136/275] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:02 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:03.594157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:03.594766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:03.601285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:03.601771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:03.602207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:03.602902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:03.603354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:03.603832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:03.749119 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:03.757364 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bcb780ab-215b-4c1b-811d-9ee58d5f3ff4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.167222 Oct 09 00:58:03.758119 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 140/276] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:03 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:04.796426 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:04.796943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:04.804055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:04.804700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:04.805715 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:04.806656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:04.807246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:04.807850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:04.882531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5f7fee34-ba2a-421e-b12f-f3efd78ac648 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:04.888616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f7fee34-ba2a-421e-b12f-f3efd78ac648 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:04.889093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f7fee34-ba2a-421e-b12f-f3efd78ac648 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:04.889524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f7fee34-ba2a-421e-b12f-f3efd78ac648 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:04.962851 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5f7fee34-ba2a-421e-b12f-f3efd78ac648 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Going to try to stop instance {{(pid=120632) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 09 00:58:04.966833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:04.974538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bc3b006a-8272-4291-810d-83222c7048aa tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.189803 Oct 09 00:58:04.975291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 137/277] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:04 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:05.006172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5f7fee34-ba2a-421e-b12f-f3efd78ac648 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action" status: 202 len: 0 microversion: 2.1 time: 0.128466 Oct 09 00:58:05.006788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 141/278] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:58:04 2022] POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action => generated 0 bytes in 130 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 00:58:05.014894 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:05.015510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:05.021330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:05.021808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:05.022240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:05.022920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:05.023370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:05.023823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:05.149778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:05.158267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-63ae0b6d-6adf-4b2c-9ef7-720509cff2f9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.147236 Oct 09 00:58:05.158828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 138/279] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:05 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:05.368323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:05.368848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:05.374857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:05.375333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:05.375792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:05.376475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:05.376923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:05.377344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:05.400558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-230cd4e3-21dd-469d-9f70-333e5ca4df6b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/ips" status: 200 len: 150 microversion: 2.1 time: 0.036679 Oct 09 00:58:05.401103 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 142/280] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 00:58:05 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/ips => generated 150 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:05.997596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:05.998195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:06.005087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:06.005557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:06.005987 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:06.006659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:06.007103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:06.007525 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:06.139284 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:06.147385 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5a4f1b98-867f-4fe3-bb1f-409415d491fb tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.153665 Oct 09 00:58:06.147968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 139/281] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:05 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:06.180779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:06.181536 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:06.187335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:06.187850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:06.188288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:06.188967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:06.189418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:06.189843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:06.316265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:06.324282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-02486dd8-aa21-4cbb-bc6b-7e743b14bbd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.147247 Oct 09 00:58:06.324844 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 143/282] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:06 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:07.169946 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:07.170554 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:07.176911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:07.177391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:07.177828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:07.178511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:07.178964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:07.179389 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:07.307894 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:07.315490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8d2477e1-7dda-4411-86bf-b28253303df8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.149362 Oct 09 00:58:07.316080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 140/283] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:07 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:07.348424 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:07.349024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:07.354798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:07.355267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:07.355729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:07.356399 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:07.356843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:07.357261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:07.484811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:07.492841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fd8498d0-d702-4344-a124-3a01512ba781 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.148724 Oct 09 00:58:07.493390 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 144/284] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:07 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:08.338647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:08.340221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:08.346590 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:08.348371 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:08.348814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:08.349508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:08.349963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:08.350388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:08.515231 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:08.516582 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:08.522812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:08.523294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:08.523755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:08.524661 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:08.525117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:08.525540 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:08.555553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:08.661561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:08.677024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dd97dd9c-3f4a-4e49-aab4-d46967ca113c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.337645 Oct 09 00:58:08.677024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 141/285] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:08 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:08.712514 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:08.722293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5c9c00c8-5b25-41cc-8b8c-ecd9c710f3d0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.210071 Oct 09 00:58:08.723101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 145/286] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:08 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:08.871454 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "abc87052-e51b-4588-a46f-73980b21e1e6"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:08.873835 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:08.879822 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:08.880288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:08.880708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:08.881376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:08.881821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:08.882230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:09.696077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:09.696679 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:09.702815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:09.703321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:09.703792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:09.704473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:09.704922 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:09.705338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:09.838241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:09.846052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cb15af37-e272-4f16-af06-efc52e414500 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.153765 Oct 09 00:58:09.846612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 146/287] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:09 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:09.851289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:09.851905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:09.857491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:09.857983 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:09.858418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:09.859097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:09.859550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:09.859998 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:09.987435 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:09.995504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-052bd188-0d4e-4d88-b87c-a08f3a506d50 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.147927 Oct 09 00:58:09.996099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 147/288] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:09 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:10.869545 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:10.870156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:10.876183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:10.876645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:10.877073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:10.877746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:10.878750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:10.879171 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:11.007510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:11.015389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e1e1101b-780d-449b-ac77-daeb096780f1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.149628 Oct 09 00:58:11.015973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 148/289] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:10 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:11.020259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:11.020859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:11.026364 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:11.026838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:11.027269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:11.027972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:11.028423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:11.028896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:11.161697 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:11.169847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-50a06ce5-5796-4e9f-a2cb-bb6742b99280 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.152959 Oct 09 00:58:11.170400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 149/290] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:11 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:11.797513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f46f75a1-c42f-4de0-947b-fcb80c8eaa90 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.929744 Oct 09 00:58:11.798176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 142/291] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 00:58:08 2022] POST /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments => generated 194 bytes in 2931 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:12.040887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:12.041840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:12.047795 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:12.048278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:12.048706 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:12.049510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:12.049853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:12.050566 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:12.188734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:12.199575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:12.200581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:12.200829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8e2ba105-00a8-4a56-be0e-5eb264a6f845 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.165481 Oct 09 00:58:12.201589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 150/292] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:12 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:12.209028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:12.209615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:12.210266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:12.211031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:12.211555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:12.212082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:12.338895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:12.346937 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d2ab7285-e3eb-462d-92e4-1a36603a6719 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.154502 Oct 09 00:58:12.347492 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 143/293] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:12 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:13.227321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:13.228104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:13.234702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:13.235283 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:13.235818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:13.236567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:13.237218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:13.237704 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:13.371664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:13.372286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:13.386539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:13.387325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:13.387869 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:13.388920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:13.389472 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:13.390004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:13.390965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:13.399577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b3a82445-5052-4220-84b3-c42a447f3542 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.178219 Oct 09 00:58:13.400168 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 151/294] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:13 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:13.532060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:13.540343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3fa5b6a5-cf2e-4e47-ba71-9b916ba2e6e6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.173537 Oct 09 00:58:13.547225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 144/295] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:13 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:14.423384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:14.423959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:14.430467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:14.430940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:14.431369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:14.432063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:14.432512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:14.432932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:14.560794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:14.564369 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:14.564988 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:14.568987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-61230184-1b02-48b0-971c-2964fa2ae152 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.149343 Oct 09 00:58:14.569563 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 152/296] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:14 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:14.571298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:14.571833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:14.572270 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:14.572967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:14.573450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:14.573873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:14.718522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:14.727288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-643a27ae-658f-4a9e-98b5-25b4f2f15fac tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.167447 Oct 09 00:58:14.727884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 145/297] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:14 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:15.592248 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:15.592798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:15.599438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:15.599920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:15.600359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:15.601035 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:15.601483 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:15.601902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:15.732764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:15.740505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-dde9cc57-0831-43b3-b13c-30f456afa86f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.152174 Oct 09 00:58:15.741960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 153/298] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:15 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:15.750986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:15.751602 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:15.758482 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:15.759011 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:15.759394 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:15.760142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:15.760588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:15.761009 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:15.915066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:15.923076 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-66d29496-8cbb-4d6e-9197-418a596f9cbf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.176093 Oct 09 00:58:15.923644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 146/299] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:15 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:16.765826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:16.766426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:16.773397 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:16.773930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:16.774371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:16.775054 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:16.775506 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:16.775962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:16.905552 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:16.913554 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-448d6c1a-a01b-472d-84b7-db15e7eda0d5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.151598 Oct 09 00:58:16.914113 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 154/300] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:16 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:16.947348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:16.947997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:16.954840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:16.955440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:16.956073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:16.957084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:16.957626 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:16.958051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:16.962015 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:17.099274 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:17.108272 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-22faae99-527f-4deb-8b78-ba4aa01716a7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.165248 Oct 09 00:58:17.108967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 147/301] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:16 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:17.937866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:17.938590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:17.945348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:17.946291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:17.946723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:17.947405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:17.947896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:17.948314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:18.089058 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:18.096773 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2121032c-c46e-4235-832f-4eebd0e6b72d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.164237 Oct 09 00:58:18.097334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 155/302] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:17 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:18.131865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:18.132659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:18.139338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:18.139874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:18.140312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:18.141026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:18.141545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:18.141972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:18.280942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:18.290016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5151b166-476f-4267-b459-8dc2b61fec14 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.162033 Oct 09 00:58:18.290567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 148/303] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:18 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:19.120015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:19.121216 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:19.127433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:19.127953 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:19.128389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:19.129667 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:19.130120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:19.130544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:19.313643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:19.314777 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:19.325689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:19.326176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:19.326606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:19.327305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:19.327790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:19.328822 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:19.342720 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:19.373211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-63af860b-b072-4d44-8f8f-226736ea51ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.256855 Oct 09 00:58:19.375121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 156/304] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:19 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:19.497358 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:19.506620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-86f02c3b-cee2-4733-9ea6-58779bafc19c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.197272 Oct 09 00:58:19.507497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 149/305] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:19 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:20.402989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:20.402989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:20.409119 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:20.409599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:20.410065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:20.410749 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:20.411221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:20.411657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:20.469409 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-02b19bfa-b3c7-498d-a305-d62f4230601a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: .version_select at 0x7f7268b70430>, body: {"keypair": {"name": "tempest-keypair-1986031973", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAB71WqgcW11C8D1U59Km6HA88dUa4IeZHy5gFNQNJEu/JAWcKIb6wRhzBuDIgs6b3UNCYTeW/BE/3RgwlgkTsh7dr8LCbqGTmf+Mke+sgvd7tASvhXZH8d1PUra0P23QA=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:20.485716 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-02b19bfa-b3c7-498d-a305-d62f4230601a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:58:20.490502 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-02b19bfa-b3c7-498d-a305-d62f4230601a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:58:20.518449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-02b19bfa-b3c7-498d-a305-d62f4230601a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.052346 Oct 09 00:58:20.519047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 150/306] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 00:58:20 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:58:20.532456 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:20.533144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:20.547798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:20.547798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:20.547798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:20.547798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:20.548235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:20.548235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:20.561914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:20.570016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5f3ec78f-9336-40bf-8f1d-efcd2e7735f0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.172891 Oct 09 00:58:20.570841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 157/307] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:20 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:20.780806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:20.791107 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3eb18b06-748f-46c8-8e31-eb0b18974b46 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.263824 Oct 09 00:58:20.791792 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 151/308] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:20 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:21.593707 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:21.594355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:21.600415 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:21.600892 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:21.601334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:21.602026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:21.602531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:21.602903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:21.802535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:21.811361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3485dbfa-10d8-4e4e-9fad-5865c3b3804b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.221415 Oct 09 00:58:21.818814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:21.819489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:21.826650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 158/309] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:21 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:21.828943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:21.829507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:21.830011 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:21.830773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:21.831352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:21.831993 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:21.994151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:22.008136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-748eda87-3d9a-43d7-9411-a543d6de1056 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.197138 Oct 09 00:58:22.008720 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 152/310] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:21 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:22.129726 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1477995665", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ef6b019b-3ff0-42c3-b2bd-354f2c856ccc"}], "security_groups": [{"name": "tempest-securitygroup--1657261559"}], "key_name": "tempest-keypair-1986031973", "user_data": "}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:22.836944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:22.837715 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:22.859719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:22.864507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:22.866595 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:22.868749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:22.871352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:22.872403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.053709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:23.061798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-455e7a00-94ee-4f12-ab5e-744ca9e538d9 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1500 microversion: 2.1 time: 0.231327 Oct 09 00:58:23.062388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 153/311] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:22 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1500 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:23.075929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:23.075929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:23.090662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.091098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.091537 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.092260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.092720 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.093148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.250486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:23.259699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b88d2fa6-a8f7-4253-9330-68b902ef9802 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.196178 Oct 09 00:58:23.260256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 154/312] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:23 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:23.264981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6c5f5598-1622-4c04-81ce-64238f6b81ac"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:23.267297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:23.273458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.274093 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.275031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.276013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.276461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.277072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.303546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] HTTP exception thrown: Cannot 'attach_volume' instance a1b50472-3065-47ec-85b8-f52e36a9374c while it is in vm_state rescued Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Request method failure captured: Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/os-volume_attachments HTTP/1.1 Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 74 Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwpCa7qTsGCbgYzI3hTp0XV4aX7UY9kYcqRmcLmnNuVu0pwn81JmE0Q7Hu-M48YX0Ivzl80hJOivFomkM56cwHMuXcUttfPVYuxDmo1TXJ7o2oEOERlIn_KBndSYXeiVrB5ti0ZjjXNGyRZ2bEWEYWMfg6VZDBo93QOHqW60b7MagU Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 00:58:23.305251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 438446d1e6b3402bb18b547c089391f8 Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {"volumeAttachment": {"volumeId": "6c5f5598-1622-4c04-81ce-64238f6b81ac"}} Oct 09 00:58:23.306813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Cannot 'attach_volume' instance a1b50472-3065-47ec-85b8-f52e36a9374c while it is in vm_state rescued Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'server_id': 'a1b50472-3065-47ec-85b8-f52e36a9374c', 'body': {'volumeAttachment': {'volumeId': '6c5f5598-1622-4c04-81ce-64238f6b81ac'}}} Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance a1b50472-3065-47ec-85b8-f52e36a9374c in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:58:23.307615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance a1b50472-3065-47ec-85b8-f52e36a9374c while it is in vm_state rescued Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:58:23.308746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance a1b50472-3065-47ec-85b8-f52e36a9374c while it is in vm_state rescued Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Returning 409 to user: Cannot 'attach_volume' instance a1b50472-3065-47ec-85b8-f52e36a9374c while it is in vm_state rescued {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-557c07ed-929a-4a79-95c8-8d31aa0c22e8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.045449 Oct 09 00:58:23.309829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 155/313] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 00:58:23 2022] POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/os-volume_attachments => generated 152 bytes in 46 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 09 00:58:23.316653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:23.317172 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:23.322746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.323217 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.323677 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.324353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.324800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.325217 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.395334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a612ebf9-fdbf-49d2-a26c-e94d5d7cadf6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/action" status: 202 len: 0 microversion: 2.1 time: 0.082295 Oct 09 00:58:23.400036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 156/314] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:58:23 2022] POST /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c/action => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:58:23.405489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:23.406460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:23.412600 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.413079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.413507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.414187 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:23.414632 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:23.415231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:23.574658 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:23.583473 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cae0fc37-ec65-42e5-a668-8b6abfd1f045 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1508 microversion: 2.1 time: 0.182218 Oct 09 00:58:23.584060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 157/315] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:23 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1508 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:24.202738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] validate_networks() for [('ef6b019b-3ff0-42c3-b2bd-354f2c856ccc', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:58:24.283530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:24.285336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:24.292196 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:24.292694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:24.293103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:24.293800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:24.295198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:24.295198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:24.406959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=50 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:24.503060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:24.511632 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e2b6032b-11f8-46c9-b3fd-08ab5127c885 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.232622 Oct 09 00:58:24.524112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 158/316] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:24 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:24.606555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:24.607169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:24.613062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:24.613544 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:24.614171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:24.614878 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:24.618053 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:58:24.618655 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:58:24.619125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:58:24.619538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:58:24.619978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:58:24.623355 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:24.624057 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:24.629883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:24.630518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:24.631050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:24.631742 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:24.632340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:24.632777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:24.651541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:58:24.657271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:58:24.674402 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:58:24.674540 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:58:24.740280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3284f9bb-e0a4-4c5c-94c3-d2c3374da2d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.614826 Oct 09 00:58:24.750241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 159/317] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:58:22 2022] POST /compute/v2.1/servers => generated 402 bytes in 2625 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 09 00:58:24.763620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ab3039b8-3b0a-4ee5-8a98-4d6430a2d75b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:24.764905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ab3039b8-3b0a-4ee5-8a98-4d6430a2d75b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:24.777496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:24.786939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-96e6d730-96c7-4086-abec-dc7b10fe3ad3 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1508 microversion: 2.1 time: 0.184111 Oct 09 00:58:24.788165 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 159/318] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:24 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1508 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:24.825991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ab3039b8-3b0a-4ee5-8a98-4d6430a2d75b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:24.836052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ab3039b8-3b0a-4ee5-8a98-4d6430a2d75b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1242 microversion: 2.1 time: 0.077061 Oct 09 00:58:24.836921 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 160/319] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:24 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1242 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:25.537968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:25.538875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:25.545717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:25.546655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:25.547403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:25.548304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:25.548840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:25.549480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:25.693894 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:25.702146 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1df6b739-dbda-4f9c-9d0a-6914405639de tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.169337 Oct 09 00:58:25.702988 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 160/320] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:25 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:25.810736 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:25.811463 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:25.820457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:25.820957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:25.821398 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:25.822225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:25.822692 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:25.823123 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:25.866435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:25.867088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:25.877912 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:25.878607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:25.879041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:25.879944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:25.880404 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:25.880858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:25.987795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:26.006928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1a336196-6d0f-4acc-9374-8e584f1ed4b3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1238 microversion: 2.1 time: 0.146039 Oct 09 00:58:26.008818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 161/321] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:25 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1238 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:26.115910 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:26.123767 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-12fb498f-740f-488a-9809-ede346dd2a20 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1508 microversion: 2.1 time: 0.316661 Oct 09 00:58:26.124311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 161/322] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:25 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1508 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:26.726911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:26.727509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:26.734028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:26.734500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:26.734928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:26.735612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:26.736066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:26.736478 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:26.868391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:26.878774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f387ad23-e9bf-4878-8a97-cd0e23607363 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.155561 Oct 09 00:58:26.879348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 162/323] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:26 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:27.046828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:27.047442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:27.053579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:27.054066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:27.054504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:27.055199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:27.055673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:27.056095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:27.126856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:27.135081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4303c7d9-df2a-4344-ad9e-917c33a297ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1294 microversion: 2.1 time: 0.092306 Oct 09 00:58:27.138802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 162/324] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:27 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1294 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:27.150037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:27.150810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:27.157812 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:27.158298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:27.158885 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:27.159827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:27.160287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:27.160774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:27.358759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:27.370361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3ed2b573-cd01-407d-8c9f-79bb0dda4406 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1514 microversion: 2.1 time: 0.223676 Oct 09 00:58:27.370948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 163/325] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:27 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1514 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:27.902637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:27.903409 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:27.910500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:27.910985 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:27.911420 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:27.912149 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:27.912785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:27.913212 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:28.051349 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:28.054802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:28.060928 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0142220a-1b94-438c-bf4a-f47c9f4e755c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1723 microversion: 2.1 time: 0.162427 Oct 09 00:58:28.061525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 163/326] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:27 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:28.160557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:28.161235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:28.168563 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:28.171160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:28.171706 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:28.172195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:28.173302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:28.173900 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:28.175239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:28.183773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:28.397701 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:28.407434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-002f4c65-39d8-4a84-88a6-4195f6cee9ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.253014 Oct 09 00:58:28.408119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 164/327] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:28 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:28.791469 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-81c630dd-1c1b-4d67-8f31-3ed49eb73263 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'create', calling method: .version_select at 0x7f72697af940>, body: {"keypair": {"name": "tempest-keypair-2109480898", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBm3XxAp4q3tqQ8HznHZQKVjKJAds2CNmjx3JqjeDj3CS7/eaHmeWIF6LLhjM2p2rdOKYEekx13Hp4cgTUwToF84rkwHZT9y+8yV2Z5cSPFX16MdSoQQ2FKJmoAGkTugow=="}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:28.808547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-81c630dd-1c1b-4d67-8f31-3ed49eb73263 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for project f68941040b3049bd9c62e2fc061d94c1. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:58:28.813231 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-81c630dd-1c1b-4d67-8f31-3ed49eb73263 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for user 438446d1e6b3402bb18b547c089391f8 and project f68941040b3049bd9c62e2fc061d94c1. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:58:28.838857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-81c630dd-1c1b-4d67-8f31-3ed49eb73263 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.051445 Oct 09 00:58:28.839634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 164/328] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 00:58:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:28.988371 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-eb8c9476-00a3-4d35-ae02-9bcc54c368d9 req-aa133384-294a-47a2-844e-5bc5eb89ea30 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9988ceee-fd6b-43f1-ae12-74b98a9f128a", "tag": "9f833987-35a7-48c2-8720-3ab67f5287e5"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:29.004775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-eb8c9476-00a3-4d35-ae02-9bcc54c368d9 req-aa133384-294a-47a2-844e-5bc5eb89ea30 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.005710 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-eb8c9476-00a3-4d35-ae02-9bcc54c368d9 req-aa133384-294a-47a2-844e-5bc5eb89ea30 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.006588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-eb8c9476-00a3-4d35-ae02-9bcc54c368d9 req-aa133384-294a-47a2-844e-5bc5eb89ea30 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.026825 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-eb8c9476-00a3-4d35-ae02-9bcc54c368d9 req-aa133384-294a-47a2-844e-5bc5eb89ea30 service nova] Creating event network-changed:9f833987-35a7-48c2-8720-3ab67f5287e5 for instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a on hv-cinder-maste Oct 09 00:58:29.037073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-eb8c9476-00a3-4d35-ae02-9bcc54c368d9 req-aa133384-294a-47a2-844e-5bc5eb89ea30 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052698 Oct 09 00:58:29.038510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 165/329] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:58:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:29.084911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:29.085626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:29.100680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.101262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.101732 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.102878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.103335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.104339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.360924 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:29.374010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-68521f7d-2894-4953-a7c4-49f1e915a81e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1700 microversion: 2.1 time: 0.293460 Oct 09 00:58:29.374845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 165/330] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:29 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1700 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:29.399335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7a3adf1e-2c65-4cd3-ae7b-083bd8bf672e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:29.408479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7a3adf1e-2c65-4cd3-ae7b-083bd8bf672e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.409800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7a3adf1e-2c65-4cd3-ae7b-083bd8bf672e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.410901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7a3adf1e-2c65-4cd3-ae7b-083bd8bf672e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.435209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:29.436200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:29.448269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.452127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.452958 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.453770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.458674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.459540 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.518716 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7a3adf1e-2c65-4cd3-ae7b-083bd8bf672e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Going to try to start instance {{(pid=120633) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 09 00:58:29.578629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7a3adf1e-2c65-4cd3-ae7b-083bd8bf672e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action" status: 202 len: 0 microversion: 2.1 time: 0.184822 Oct 09 00:58:29.579814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 166/331] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:58:29 2022] POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:58:29.589742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:29.591760 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:29.605342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.606016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.606816 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.608335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:29.608964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:29.609552 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:29.931987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:29.941495 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f7e38626-dcec-4e60-b454-62b513d7c697 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.514169 Oct 09 00:58:29.942072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 166/332] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:29 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:29.946805 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-635917312", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1094622065", "security_groups": [{"name": "tempest-secgroup-smoke-606774372"}], "block_device_mapping_v2": [{"uuid": "b9013137-781e-4610-982f-a64062ab5a38", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "51f66c80-6e94-4ac9-ad00-6dd9246260d1"}]}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:30.066842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] validate_networks() for [('51f66c80-6e94-4ac9-ad00-6dd9246260d1', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:58:30.141307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:30.150834 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f5d1d49b-481a-4746-bb20-440e63c3e667 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1709 microversion: 2.1 time: 0.566929 Oct 09 00:58:30.152067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 167/333] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:29 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1709 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:30.467964 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:58:30.468583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:58:30.469243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:58:30.469594 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:58:30.470015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:58:30.480681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:30.481460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:30.482066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:30.482975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:30.483609 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:30.484228 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:30.505779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:58:30.510380 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for user 04bc22d558c2466ba43a456d08bdd6bd and project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:58:30.579520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:58:30.581799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b9013137-781e-4610-982f-a64062ab5a38',volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:58:30.742737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-288f6e06-0d64-4ef4-8fc1-7ffc5b4aaee2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.799263 Oct 09 00:58:30.750104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 167/334] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:58:29 2022] POST /compute/v2.1/servers => generated 405 bytes in 805 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:58:30.756949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5adc8efa-e052-4e46-b5ed-7f4a9fceb3eb tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:30.758020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5adc8efa-e052-4e46-b5ed-7f4a9fceb3eb tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:30.760484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=4 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:30.857816 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5adc8efa-e052-4e46-b5ed-7f4a9fceb3eb tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:30.876700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5adc8efa-e052-4e46-b5ed-7f4a9fceb3eb tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1094 microversion: 2.1 time: 0.124589 Oct 09 00:58:30.877598 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 168/335] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:30 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1094 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:30.967260 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:30.967260 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:30.983779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:30.984307 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:30.984745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:30.986091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:30.986570 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:30.987007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:31.182479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:31.184333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:31.201095 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:31.201958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:31.202595 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:31.205472 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:31.206536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:31.207178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:31.256870 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:31.268801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a4b192a1-fb4d-4849-98ab-72c0db815901 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.307126 Oct 09 00:58:31.271231 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 168/336] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:30 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:31.284528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1515406684", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "dd937102-41e7-4629-a161-cb0fb39a0995"}], "security_groups": [{"name": "tempest-securitygroup--465916780"}], "key_name": "tempest-keypair-2109480898", "user_data": "}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:31.428573 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] validate_networks() for [('dd937102-41e7-4629-a161-cb0fb39a0995', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:58:31.573751 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:31.603018 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0b789a94-bd2f-4aab-a463-d2232b4658aa tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1709 microversion: 2.1 time: 0.429568 Oct 09 00:58:31.622942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 169/337] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:31 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1709 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:31.900102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:31.900776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:31.907984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:31.908205 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:31.908823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:31.909743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:31.910350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:31.910943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.019922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:32.041020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8c91d151-6a07-4138-a724-7daf6862cc71 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1192 microversion: 2.1 time: 0.144900 Oct 09 00:58:32.042293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 170/338] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:31 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1192 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:32.061822 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:58:32.062894 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:58:32.063526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:58:32.064109 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:58:32.064678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:58:32.077166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:32.077963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:32.078621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.079880 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:32.080647 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:32.082001 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.104841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for project f68941040b3049bd9c62e2fc061d94c1. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:58:32.110354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Getting quotas for user 438446d1e6b3402bb18b547c089391f8 and project f68941040b3049bd9c62e2fc061d94c1. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:58:32.128709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:58:32.131145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:58:32.191177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1ef069dd-8591-4850-97a9-0a54461f3b70 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.917744 Oct 09 00:58:32.192338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 169/339] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:58:31 2022] POST /compute/v2.1/servers => generated 408 bytes in 920 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:58:32.209995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-49e73c66-ff7b-473d-a706-fe593829f63a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:32.211789 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-49e73c66-ff7b-473d-a706-fe593829f63a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:32.300884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-49e73c66-ff7b-473d-a706-fe593829f63a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:32.310115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:32.310827 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:32.325114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-49e73c66-ff7b-473d-a706-fe593829f63a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1246 microversion: 2.1 time: 0.120092 Oct 09 00:58:32.343632 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 171/340] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:32 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1246 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:32.343845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:32.343845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:32.343845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.343845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:32.344146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:32.344146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.514180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:32.533152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6e938cab-dc04-46eb-8cea-bceb3b787cf3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.240540 Oct 09 00:58:32.536157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 170/341] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:32 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:32.631336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:32.632264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:32.638526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:32.639011 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:32.639801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.640853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:32.641662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:32.642335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:32.886181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:32.894578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5e6a3735-770b-4419-84c0-68524d35e01a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1709 microversion: 2.1 time: 0.268116 Oct 09 00:58:32.895142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 172/342] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:32 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1709 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:33.069374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:33.070538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:33.079865 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.080647 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.081968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.083038 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.083863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.084444 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.172546 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:33.188115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-22202eb3-e57f-4b3a-bd17-4db6b1f41e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1210 microversion: 2.1 time: 0.126438 Oct 09 00:58:33.204133 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 171/343] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:33 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1210 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:33.356994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:33.357620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:33.370640 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.371633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.372489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.373440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.376119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.376874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.473010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:33.481048 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5422a59d-105c-4b65-ad19-487bfb7fda38 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1298 microversion: 2.1 time: 0.127897 Oct 09 00:58:33.481538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 173/344] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:33 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1298 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:33.557180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:33.558193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:33.570987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.572701 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.573159 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.573883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.575173 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.575872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.750539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:33.760286 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-47c6e87d-ad16-40f2-aa77-97ccac1f165a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.207374 Oct 09 00:58:33.761776 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 172/345] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:33 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:33.923275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:33.929456 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:33.941014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.941664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.942515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:33.943578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:33.944838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:33.946968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.212898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:34.213502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:34.217126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:34.220651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:34.221371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:34.222039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.222808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:34.223622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:34.224257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.226263 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e84b16f7-a3b0-480f-a395-8781a2b22741 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1713 microversion: 2.1 time: 0.306728 Oct 09 00:58:34.227146 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 174/346] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:33 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1713 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:34.392027 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:34.405007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2cbbafca-f8e7-4aed-b77d-56b30e56db46 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1277 microversion: 2.1 time: 0.196140 Oct 09 00:58:34.406674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 173/347] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:34 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1277 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:34.505058 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:34.505673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:34.516516 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:34.523433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:34.524262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.525809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:34.526457 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:34.527108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.693771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:34.702725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3060d389-88fa-4398-86fc-d3c991e51a44 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1298 microversion: 2.1 time: 0.201338 Oct 09 00:58:34.704581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 175/348] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:34 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1298 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:34.786671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:34.787666 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:34.812671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:34.812814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:34.813221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.814476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:34.814592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:34.815660 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:34.984373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:34.993842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-61e29bea-dcf0-41c9-8f52-d786407654f3 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.211038 Oct 09 00:58:34.994691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 174/349] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:34 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:35.429718 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:35.430636 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:35.436681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:35.437097 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:35.437530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:35.438218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:35.438670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:35.439093 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:35.729251 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:35.729799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:35.736984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:35.740973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:35.741709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:35.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:35.742859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:35.743316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:35.744725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:35.746356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ce65d81d-8a26-4621-aca6-a6a926bfa2b0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.320436 Oct 09 00:58:35.747060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 176/350] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:35 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:35.825423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-942a181d-2b8f-439a-a807-918a45d4c573 req-8fdf3154-26f7-450b-a40d-7cffd5b1bf6e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "841a7957-ee73-4765-a850-406b1e720612", "tag": "c0379d91-6e4b-4a6c-895c-fc71a455dbb1"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:35.825835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:35.834328 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-54bd3537-042a-4213-89af-72c81df95709 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1304 microversion: 2.1 time: 0.109097 Oct 09 00:58:35.835212 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 175/351] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:35 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1304 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:35.837647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-942a181d-2b8f-439a-a807-918a45d4c573 req-8fdf3154-26f7-450b-a40d-7cffd5b1bf6e service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:35.838912 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-942a181d-2b8f-439a-a807-918a45d4c573 req-8fdf3154-26f7-450b-a40d-7cffd5b1bf6e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:35.839524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-942a181d-2b8f-439a-a807-918a45d4c573 req-8fdf3154-26f7-450b-a40d-7cffd5b1bf6e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:35.867963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-942a181d-2b8f-439a-a807-918a45d4c573 req-8fdf3154-26f7-450b-a40d-7cffd5b1bf6e service nova] Creating event network-changed:c0379d91-6e4b-4a6c-895c-fc71a455dbb1 for instance 841a7957-ee73-4765-a850-406b1e720612 on hv-cinder-maste Oct 09 00:58:35.880967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-942a181d-2b8f-439a-a807-918a45d4c573 req-8fdf3154-26f7-450b-a40d-7cffd5b1bf6e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061729 Oct 09 00:58:35.880967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 177/352] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:58:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:58:36.017832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:36.019449 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:36.029983 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:36.030750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:36.031190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:36.032094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:36.032867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:36.033294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:36.199075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:36.208187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9d936a72-6eea-42ac-ab15-42c5d07c7064 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.194524 Oct 09 00:58:36.208843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 176/353] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:36 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:36.771308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:36.772027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:36.778149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:36.778628 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:36.779064 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:36.779780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:36.780233 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:36.780911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:36.858626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:36.859361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:36.865815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:36.866305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:36.866746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:36.867442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:36.867931 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:36.868359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:36.915325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:36.923118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d455ca0c-efe2-46cd-b7ca-8f62deb68ae4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.157040 Oct 09 00:58:36.923699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 178/354] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:36 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:37.002069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:37.010316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9f892b6d-3cd2-45f7-ad90-6ef931dda110 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.155834 Oct 09 00:58:37.011112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 177/355] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:36 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:37.231059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:37.231719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:37.238836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:37.239260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:37.239731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:37.240431 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:37.240883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:37.241308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:37.368793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:37.376295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ab1d4b4e-e532-494a-8f0e-d2c89b2dc0ea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.149120 Oct 09 00:58:37.376866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 179/356] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:37 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:37.944464 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:37.945116 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:37.951115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:37.951616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:37.952065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:37.952761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:37.953215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:37.953642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:38.033517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:38.034751 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:38.041245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:38.041735 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:38.042179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:38.042873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:38.043325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:38.043781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:38.089198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:38.097568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-dda1c036-e9f6-4a03-8cb3-a9c4db370d4f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.156802 Oct 09 00:58:38.098533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 178/357] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:37 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:38.286866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:38.294468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e5de55c8-4b19-4f0c-9c63-1bcda44e4394 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.265022 Oct 09 00:58:38.295773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 180/358] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:38 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:38.326333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-f12cd782-c9e1-4a13-a1fb-672a05271975 req-e41c738b-8c51-46e6-837c-05878da97ef2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9988ceee-fd6b-43f1-ae12-74b98a9f128a", "name": "network-vif-plugged", "status": "completed", "tag": "9f833987-35a7-48c2-8720-3ab67f5287e5"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:38.337333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f12cd782-c9e1-4a13-a1fb-672a05271975 req-e41c738b-8c51-46e6-837c-05878da97ef2 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:38.337987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f12cd782-c9e1-4a13-a1fb-672a05271975 req-e41c738b-8c51-46e6-837c-05878da97ef2 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:38.338575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f12cd782-c9e1-4a13-a1fb-672a05271975 req-e41c738b-8c51-46e6-837c-05878da97ef2 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:38.358680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-f12cd782-c9e1-4a13-a1fb-672a05271975 req-e41c738b-8c51-46e6-837c-05878da97ef2 service nova] Creating event network-vif-plugged:9f833987-35a7-48c2-8720-3ab67f5287e5 for instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a on hv-cinder-maste Oct 09 00:58:38.366199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-f12cd782-c9e1-4a13-a1fb-672a05271975 req-e41c738b-8c51-46e6-837c-05878da97ef2 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043323 Oct 09 00:58:38.367013 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 179/359] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:58:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:38.403479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:38.405071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:38.415631 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:38.415748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:38.416200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:38.417197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:38.418156 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:38.418863 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:38.827939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:38.835676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7c4e6199-e2fc-4a8b-aa7c-882a5653bf2b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.439599 Oct 09 00:58:38.836238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 181/360] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:38 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:39.121645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:39.122221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:39.127972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:39.133778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.134491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.135085 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:39.135973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.136627 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.137302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:39.289937 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-a825f034-c763-47bc-be2c-bb688572af64 req-a1cbed16-e31b-4e53-bf24-8835869b0bd6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50224067-d3e0-4567-bc63-31ace3a211c6", "tag": "af11de25-9b71-43af-8ce0-051a32ec945c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:39.305842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:39.308663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a825f034-c763-47bc-be2c-bb688572af64 req-a1cbed16-e31b-4e53-bf24-8835869b0bd6 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.309162 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a825f034-c763-47bc-be2c-bb688572af64 req-a1cbed16-e31b-4e53-bf24-8835869b0bd6 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.309798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a825f034-c763-47bc-be2c-bb688572af64 req-a1cbed16-e31b-4e53-bf24-8835869b0bd6 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:39.315207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-df36601b-8967-4449-a98d-1e730f0d615c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.198010 Oct 09 00:58:39.315918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 180/361] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:39 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:39.320673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:39.321278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:39.324152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-a825f034-c763-47bc-be2c-bb688572af64 req-a1cbed16-e31b-4e53-bf24-8835869b0bd6 service nova] Creating event network-changed:af11de25-9b71-43af-8ce0-051a32ec945c for instance 50224067-d3e0-4567-bc63-31ace3a211c6 on hv-cinder-maste Oct 09 00:58:39.327304 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.328124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.329047 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:39.329994 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.330747 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.331146 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-a825f034-c763-47bc-be2c-bb688572af64 req-a1cbed16-e31b-4e53-bf24-8835869b0bd6 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047980 Oct 09 00:58:39.331465 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:39.331962 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 182/362] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:58:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:58:39.459850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:39.467605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-037a8a0a-0c2f-4016-8641-3b22f98a515f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.150571 Oct 09 00:58:39.468168 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 181/363] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:39 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:39.861451 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:39.862973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:39.870783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.871511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.872221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:39.883886 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:39.885895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:39.887907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:40.093744 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:40.096004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:40.102465 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-71af9e79-3b6c-4add-8c20-8bd38a78a584 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1368 microversion: 2.1 time: 0.247180 Oct 09 00:58:40.103029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 183/364] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:39 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1368 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:40.342309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:40.342919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:40.348990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:40.349475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:40.349912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:40.350598 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:40.351050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:40.351475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:40.484278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:40.491738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:40.492341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:40.492785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1c5fbcd4-5ef5-49ef-a7fc-c02566e2d031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.159226 Oct 09 00:58:40.493371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 182/365] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:40 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:40.499660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:40.500256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:40.500691 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:40.501386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:40.501838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:40.502275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:40.633235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:40.640732 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-455df068-3bb4-46f2-a9ce-41d6302359c1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.154358 Oct 09 00:58:40.641292 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 184/366] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:40 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:41.125118 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:41.125862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:41.133058 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.133902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.134505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.135357 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.136020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.136450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.164152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:41.342909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:41.352403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0e1bae04-c7d3-4dc2-ae96-df392730c3ae tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1559 microversion: 2.1 time: 0.231569 Oct 09 00:58:41.353374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 183/367] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:41 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1559 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:41.384067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:41.385068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:41.392521 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.393738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.394384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.395282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.396194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.396825 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.482205 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a095ee06-6706-4c8e-a4e6-117fe24c57a8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.104032 Oct 09 00:58:41.482786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 185/368] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 00:58:41 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/os-interface => generated 285 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:58:41.516648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:41.517508 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:41.524064 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.524546 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.524983 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.525672 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.526121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.526542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.663899 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:41.664717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:41.670740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.671222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.671681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.672370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:41.672829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:41.673252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:41.768793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:41.777102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9bc1724b-298d-4cd6-aeb9-5e74d4e3d8b6 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.264283 Oct 09 00:58:41.788128 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 184/369] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:41 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:41.876260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:41.884549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1d125f35-6b48-4045-97b9-a716eecb0041 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.224926 Oct 09 00:58:41.885118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 186/370] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:41 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:42.519316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-b0cd7f05-131d-4832-831b-9248742901d2 req-8b9a494e-e6b3-4ea5-9cf9-9ea5474a0f1b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9988ceee-fd6b-43f1-ae12-74b98a9f128a", "tag": "9f833987-35a7-48c2-8720-3ab67f5287e5"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:42.531315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:42.532821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=24 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:42.535216 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b0cd7f05-131d-4832-831b-9248742901d2 req-8b9a494e-e6b3-4ea5-9cf9-9ea5474a0f1b service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:42.535981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b0cd7f05-131d-4832-831b-9248742901d2 req-8b9a494e-e6b3-4ea5-9cf9-9ea5474a0f1b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:42.537911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b0cd7f05-131d-4832-831b-9248742901d2 req-8b9a494e-e6b3-4ea5-9cf9-9ea5474a0f1b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:42.557169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-b0cd7f05-131d-4832-831b-9248742901d2 req-8b9a494e-e6b3-4ea5-9cf9-9ea5474a0f1b service nova] Creating event network-changed:9f833987-35a7-48c2-8720-3ab67f5287e5 for instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a on hv-cinder-maste Oct 09 00:58:42.564036 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-b0cd7f05-131d-4832-831b-9248742901d2 req-8b9a494e-e6b3-4ea5-9cf9-9ea5474a0f1b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049066 Oct 09 00:58:42.565565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 185/371] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:58:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:42.800784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:42.801389 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:42.803670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=4 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:42.810038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:42.810529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:42.810961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:42.811879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:42.812333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:42.812755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:42.909540 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:42.911299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:42.919768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:42.920745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:42.921905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:42.922908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:42.923510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:42.924339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:42.972601 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:42.981328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c02fa75d-3cc5-483b-b13f-b4c9efbccd1f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.184353 Oct 09 00:58:42.982010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 187/372] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:42 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:43.101807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:43.110558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-41f09e06-ddb3-4adb-8bd1-50e873379a06 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.207836 Oct 09 00:58:43.112072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 186/373] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:42 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:44.004184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:44.006144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:44.013812 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:44.014620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:44.015209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:44.016289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:44.016892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:44.018115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:44.136988 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:44.137683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:44.151442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:44.152507 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:44.153849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:44.154672 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:44.155788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:44.156317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:44.204607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:44.215090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-404c3fa4-8700-4953-832e-0651d7c7ac25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.215131 Oct 09 00:58:44.234425 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 188/374] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:43 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:44.384799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:44.394325 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-20d40160-1572-42ea-a141-3eb7c269393f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.262364 Oct 09 00:58:44.395516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 187/375] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:44 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:45.239754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:45.241089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:45.251304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:45.252598 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:45.253182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:45.254026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:45.254648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:45.255222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:45.400868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:45.409593 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e691d8df-5b3a-4624-8630-4d42c1f11e3e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.173502 Oct 09 00:58:45.410549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 189/376] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:45 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:45.417541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:45.418344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:45.425158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:45.425799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:45.426303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:45.427065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:45.427613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:45.428104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:45.590758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:45.599518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8cecf673-834f-48de-8164-9e4fd1eab9ba tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.185700 Oct 09 00:58:45.600180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 188/377] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:45 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:46.433703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:46.434868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:46.445855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:46.446346 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:46.446782 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:46.447483 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:46.447994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:46.452119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:46.462552 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-ac963d4c-958c-40aa-b7a0-944cdd90efef req-d2bec578-8390-434d-9e1c-2780afc50247 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "50224067-d3e0-4567-bc63-31ace3a211c6", "name": "network-vif-plugged", "status": "completed", "tag": "af11de25-9b71-43af-8ce0-051a32ec945c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:46.474158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-ac963d4c-958c-40aa-b7a0-944cdd90efef req-d2bec578-8390-434d-9e1c-2780afc50247 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:46.474629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-ac963d4c-958c-40aa-b7a0-944cdd90efef req-d2bec578-8390-434d-9e1c-2780afc50247 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:46.475493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-ac963d4c-958c-40aa-b7a0-944cdd90efef req-d2bec578-8390-434d-9e1c-2780afc50247 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:46.490127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-ac963d4c-958c-40aa-b7a0-944cdd90efef req-d2bec578-8390-434d-9e1c-2780afc50247 service nova] Creating event network-vif-plugged:af11de25-9b71-43af-8ce0-051a32ec945c for instance 50224067-d3e0-4567-bc63-31ace3a211c6 on hv-cinder-maste Oct 09 00:58:46.497794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-ac963d4c-958c-40aa-b7a0-944cdd90efef req-d2bec578-8390-434d-9e1c-2780afc50247 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038251 Oct 09 00:58:46.499307 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 189/378] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:58:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:46.625079 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:46.626940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:46.639630 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:46.640641 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:46.641457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:46.643236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:46.644283 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:46.645248 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:46.655866 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:46.668769 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bad6514a-db35-47df-8160-575fd5e7e452 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.238678 Oct 09 00:58:46.676223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 190/379] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:46 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:46.858509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:46.867758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1132dff1-c08b-46d2-8edf-b85b6e5436a7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.248768 Oct 09 00:58:46.868553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 190/380] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:46 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:47.694027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:47.694738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:47.702794 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:47.703264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:47.703713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:47.704386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:47.704873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:47.705286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:47.836350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:47.844073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6a652112-3537-46a7-a841-77678888dfa4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.154719 Oct 09 00:58:47.844629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 191/381] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:47 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:47.890993 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:47.891619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:47.897534 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:47.898016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:47.898477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:47.899172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:47.899640 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:47.900062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:48.034442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:48.042637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b36dc700-c947-4e18-be92-5b595518181b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.155475 Oct 09 00:58:48.043192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 191/382] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:47 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:48.338279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1204652769", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "networks": [{"uuid": "c27214a2-3f39-42ec-9d84-0333462c2425"}]}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:58:48.391711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] validate_networks() for [('c27214a2-3f39-42ec-9d84-0333462c2425', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:58:48.737973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:58:48.738518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:58:48.738883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:58:48.739320 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:58:48.739762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:58:48.748762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:48.749175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:48.749600 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:48.750300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:48.750744 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:48.751189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:48.768148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Getting quotas for project 5edae5583d2e4a4688bef23f3a14eda4. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:58:48.772437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Getting quotas for user adcbf206828d49b9bf2ae720f71068bc and project 5edae5583d2e4a4688bef23f3a14eda4. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:58:48.785058 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:58:48.787622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:58:48.838975 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5224cabb-0bac-442b-92eb-8631ced00471 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.504323 Oct 09 00:58:48.839628 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 192/383] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:58:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 506 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:58:48.853786 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2009b373-2016-4f0a-b587-1bd193255260 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:48.854421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2009b373-2016-4f0a-b587-1bd193255260 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:48.872405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:48.873831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:48.882912 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:48.883674 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:48.884204 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:48.884989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:48.885873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:48.886299 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:48.935751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2009b373-2016-4f0a-b587-1bd193255260 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:48.944399 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2009b373-2016-4f0a-b587-1bd193255260 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1214 microversion: 2.1 time: 0.094594 Oct 09 00:58:48.945290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 192/384] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:48 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1214 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:49.030497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:49.038564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ff0c07eb-1a52-4adf-ba0b-7ca573bdfb3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.173083 Oct 09 00:58:49.039118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 193/385] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:48 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:49.065412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:49.066026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:49.071992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:49.072485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:49.072933 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:49.074055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:49.074523 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:49.074968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:49.211007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:49.219173 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ed2f97cd-d37e-4aca-8db0-337bb795624c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.157569 Oct 09 00:58:49.219903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 193/386] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:49 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:49.970552 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:49.971230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:49.977471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:49.977944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:49.978373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:49.979044 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:49.979488 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:49.979932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:50.041698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:50.050995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dbb50e6c-29a8-4dc7-98e2-82e42fa4ab7d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.085296 Oct 09 00:58:50.051557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 194/387] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:49 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:50.062738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:50.063377 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:50.070504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:50.070982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:50.071413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:50.072325 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:50.072853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:50.073317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:50.201243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:50.209054 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bf72917a-5057-4d4a-b3ee-e9ac6e0396b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.150653 Oct 09 00:58:50.209612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 194/388] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:50 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:50.243770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:50.244378 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:50.250179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:50.250660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:50.251094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:50.251850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:50.252302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:50.252727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:50.379050 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:50.386513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0045e445-9683-4064-9a93-d58ce6728b07 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.146634 Oct 09 00:58:50.387067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 195/389] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:50 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:51.074196 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:51.075017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:51.081677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:51.082292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:51.082727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:51.083516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:51.084266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:51.084677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:51.158434 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:51.166263 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ed1a99af-6f69-44c4-8caa-63270cac267a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.095904 Oct 09 00:58:51.166825 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 195/390] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:51 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:51.232206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:51.232838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:51.239846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:51.240328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:51.240759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:51.241443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:51.241884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:51.242298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:51.398286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:51.408527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4b36aecc-a1ab-45ef-ac2a-039d2f91ad38 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.179688 Oct 09 00:58:51.409372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 196/391] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:51 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:51.412647 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:51.413256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:51.426631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:51.427121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:51.427617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:51.428319 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:51.428771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:51.429194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:51.574661 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:51.583612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6ebd9275-7df2-4022-846a-4c393748de63 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.177805 Oct 09 00:58:51.584895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 196/392] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:51 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:52.189860 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:52.190467 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:52.196363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:52.196841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:52.197276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:52.197956 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:52.198408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:52.198833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:52.263024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:52.270590 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fa65f522-a59c-47c9-8e07-070c67534ff4 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.085119 Oct 09 00:58:52.271141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 197/393] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:52 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:52.430625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:52.431262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:52.437282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:52.437765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:52.438204 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:52.439547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:52.440030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:52.440461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:52.569878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:52.577591 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5960807f-f10b-4381-a83e-fdbe5607b95d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.150822 Oct 09 00:58:52.578152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 197/394] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:52 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:52.608281 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:52.608896 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:52.615935 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:52.616435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:52.616876 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:52.617599 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:52.618058 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:52.618484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:52.745064 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:52.752737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b916bc67-15ae-4163-90c6-0317557aa760 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.148384 Oct 09 00:58:52.753300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 198/395] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:52 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:53.292930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:53.293544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:53.299507 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:53.300018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:53.300454 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:53.301137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:53.301598 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:53.302016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:53.361185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:53.368581 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fc050514-fe64-474f-89c0-77035ed0d2b6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.079486 Oct 09 00:58:53.369137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 198/396] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:53 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:53.657605 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:53.658127 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:53.664471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:53.664949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:53.665380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:53.666061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:53.666508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:53.666926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:53.775141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:53.775781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:53.781900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:53.782408 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:53.782842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:53.783538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:53.784191 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:53.784617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:53.794311 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:53.802372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-242c1661-8ef9-4bf7-9c11-68eeb06c59c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.148911 Oct 09 00:58:53.803134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 199/397] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:53 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:53.915142 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:53.923440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-675ee77f-1243-41ac-874a-6196d306dd8b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.151654 Oct 09 00:58:53.924020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 199/398] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:53 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:54.391545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:54.392622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:54.398510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:54.398986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:54.399419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:54.400124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:54.400575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:54.400994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:54.462931 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:54.471077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-31c0aa39-9320-4f92-b597-28977025e97a tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.083494 Oct 09 00:58:54.471659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 200/399] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:54 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:54.824883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:54.825499 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:54.832078 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:54.832562 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:54.832996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:54.833685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:54.834138 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:54.834662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:54.946069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:54.946670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:54.952941 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:54.953420 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:54.953858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:54.955209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:54.955686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:54.956109 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:54.963821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:54.971440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6b212875-bba7-4874-af04-9abdc0116f45 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.150303 Oct 09 00:58:54.971966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:54.972392 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 200/400] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:54 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:55.085510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:55.093094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7c30d035-7e59-4c07-8753-6fd845a0946d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.150781 Oct 09 00:58:55.093668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 201/401] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:54 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:55.493541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:55.494145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:55.500183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:55.500660 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:55.501182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:55.501855 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:55.502301 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:55.502717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:55.563623 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:55.571072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6725c60e-76c9-4d42-aec7-d60bf5503916 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.081368 Oct 09 00:58:55.571634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 201/402] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:55 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:55.993924 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:55.994519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:56.000850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:56.001350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:56.001780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:56.002461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:56.002902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:56.003318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:56.116201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:56.116909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:56.122838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:56.123321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:56.123792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:56.124487 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:56.124939 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:56.125371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:56.131990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:56.139603 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-330fd0a6-71e9-43c7-b6e7-50b8f58f4333 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.149505 Oct 09 00:58:56.140177 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 202/403] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:55 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:56.144578 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:56.256533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:56.264142 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7051437b-d6e2-43a1-8099-d69a83bd5fb1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.151833 Oct 09 00:58:56.264700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 202/404] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:56 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:56.593757 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:56.594289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:56.600212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:56.600691 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:56.601127 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:56.601812 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:56.602266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:56.602690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:56.664662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:56.672143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9f77dea0-c6c8-4122-9f9f-bba2105a7275 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1218 microversion: 2.1 time: 0.082305 Oct 09 00:58:56.673200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 203/405] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:56 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1218 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:57.162829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:57.163433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:57.169380 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:57.169853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:57.170279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:57.171012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:57.171395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:57.171835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:57.287905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:57.288555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:57.295265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:57.295858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:57.296295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:57.296989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:57.297437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:57.298466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:57.301207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:57.309192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-310eff2a-b85e-4d40-a924-8ca04c9ed088 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.150298 Oct 09 00:58:57.309750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 203/406] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:57 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:57.424742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:57.432207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5e21cb32-fd39-400b-b082-3515a99e9d16 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.148185 Oct 09 00:58:57.432765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 204/407] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:57 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:57.695070 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:57.695615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:57.701557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:57.702042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:57.702481 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:57.703170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:57.703650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:57.704617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:57.763890 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:57.771964 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e55918e1-3d92-4b75-81e1-d9df8ebe8c28 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1210 microversion: 2.1 time: 0.080764 Oct 09 00:58:57.772534 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 204/408] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:57 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1210 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:58.332037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:58.332644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:58.336019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:58.339983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:58.340455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:58.340884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:58.341560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:58.342003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:58.342415 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:58.455136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:58.455798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:58.461619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:58.462105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:58.462564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:58.463257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:58.463731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:58.464154 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:58.474560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:58.482215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a80d2160-81e5-49e8-b5b0-52a0c7c69311 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.154595 Oct 09 00:58:58.482774 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 205/409] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:58 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:58.592061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:58.600031 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-454d77a7-d864-454b-958e-3aa78c039a8a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1371 microversion: 2.1 time: 0.148712 Oct 09 00:58:58.600592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 205/410] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:58 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1371 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:58.794522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:58.795092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:58.796952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:58.797606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:58.803665 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:58.804149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:58.804581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:58.805269 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:58.805719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:58.806142 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:58.867701 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:58.875193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-96867fc3-80bd-4f47-99fd-fcac98e673a3 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1266 microversion: 2.1 time: 0.084505 Oct 09 00:58:58.875825 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 206/411] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:58 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1266 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:59.506839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:59.507378 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:59.509211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:58:59.514837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.515323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.515792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.517048 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.517501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.517920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.622797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:59.623401 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:59.630080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.630566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.631017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.631722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.632179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.632605 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.649015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:59.656985 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a37c8d8d-d8ca-4127-aff1-5d23f0c4b58b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.156166 Oct 09 00:58:59.657550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 206/412] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:59 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:58:59.762328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:59.769936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e661d0a3-40f1-464c-b811-f02fa1b1d182 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1564 microversion: 2.1 time: 0.151192 Oct 09 00:58:59.770498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 207/413] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:59 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1564 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:58:59.792248 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:59.792740 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:59.799044 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.799522 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.799984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.800670 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.801123 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.801550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.878842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-896ba1f6-7185-49ce-8fa5-8c4559cb9126 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090323 Oct 09 00:58:59.879435 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 207/414] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 00:58:59 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:58:59.897434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:58:59.898038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:58:59.903819 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.904550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.904985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.905662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:58:59.906105 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:58:59.906518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:58:59.970019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:58:59.977626 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5d3aae31-57b6-4487-84e4-79184ed533af tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1274 microversion: 2.1 time: 0.084071 Oct 09 00:58:59.990447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 208/415] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:58:59 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1274 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:00.680043 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:00.681276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:00.687551 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:00.689210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:00.689210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:00.689485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:00.689938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:00.690530 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:00.925963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:00.942259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0babf77c-fa8c-4d77-b2b3-e4b4d3bb195a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.265387 Oct 09 00:59:00.955641 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 208/416] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:00 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:01.001499 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:01.002118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:01.009452 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:01.009917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:01.010356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:01.013815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:01.014300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:01.014743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:01.030508 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-f043c190-4a2f-4c10-adfe-f417c30151d6 req-b01f3658-6281-4365-8d12-920257dad8b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50224067-d3e0-4567-bc63-31ace3a211c6", "tag": "af11de25-9b71-43af-8ce0-051a32ec945c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:01.041652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f043c190-4a2f-4c10-adfe-f417c30151d6 req-b01f3658-6281-4365-8d12-920257dad8b1 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:01.042136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f043c190-4a2f-4c10-adfe-f417c30151d6 req-b01f3658-6281-4365-8d12-920257dad8b1 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:01.042569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f043c190-4a2f-4c10-adfe-f417c30151d6 req-b01f3658-6281-4365-8d12-920257dad8b1 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:01.054105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-f043c190-4a2f-4c10-adfe-f417c30151d6 req-b01f3658-6281-4365-8d12-920257dad8b1 service nova] Creating event network-changed:af11de25-9b71-43af-8ce0-051a32ec945c for instance 50224067-d3e0-4567-bc63-31ace3a211c6 on hv-cinder-maste Oct 09 00:59:01.061017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-f043c190-4a2f-4c10-adfe-f417c30151d6 req-b01f3658-6281-4365-8d12-920257dad8b1 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034948 Oct 09 00:59:01.062244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 209/417] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:59:01.335837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:01.343826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-adb706ca-f652-4147-b902-c7ce0e74313f tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.346711 Oct 09 00:59:01.344656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 209/418] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:00 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:01.967247 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:01.971905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:01.978702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:01.979192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:01.979862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:01.981646 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:01.982313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:01.983221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:02.129100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:02.137604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3b1d6735-2d99-4ffa-afcb-64f940161a5b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.174996 Oct 09 00:59:02.138288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 210/419] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:01 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:02.366517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:02.367664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:02.377264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:02.377766 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:02.378441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:02.379471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:02.379618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:02.380037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:02.423555 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-78220c70-d2ea-4c8f-98b2-19c5280cafd2 req-9d2d0e48-68bf-4578-958f-58cb0a430c8a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f748f5a5-4a7a-48c7-8790-a5c7ed797db7", "tag": "d05ad078-2684-4d21-8255-efa3f61a8cb4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:02.443786 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-78220c70-d2ea-4c8f-98b2-19c5280cafd2 req-9d2d0e48-68bf-4578-958f-58cb0a430c8a service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:02.444835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-78220c70-d2ea-4c8f-98b2-19c5280cafd2 req-9d2d0e48-68bf-4578-958f-58cb0a430c8a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:02.446365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-78220c70-d2ea-4c8f-98b2-19c5280cafd2 req-9d2d0e48-68bf-4578-958f-58cb0a430c8a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:02.460374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-78220c70-d2ea-4c8f-98b2-19c5280cafd2 req-9d2d0e48-68bf-4578-958f-58cb0a430c8a service nova] Creating event network-changed:d05ad078-2684-4d21-8255-efa3f61a8cb4 for instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 on hv-cinder-maste Oct 09 00:59:02.465848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-78220c70-d2ea-4c8f-98b2-19c5280cafd2 req-9d2d0e48-68bf-4578-958f-58cb0a430c8a service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046171 Oct 09 00:59:02.466512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 211/420] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:59:02.496714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-10a8bfcb-c52b-43ce-8ef6-14b957f38316 req-4ecea3ed-ce61-4e2a-ab34-82d7b00e5265 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "841a7957-ee73-4765-a850-406b1e720612", "name": "network-vif-plugged", "status": "completed", "tag": "c0379d91-6e4b-4a6c-895c-fc71a455dbb1"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:02.507427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-10a8bfcb-c52b-43ce-8ef6-14b957f38316 req-4ecea3ed-ce61-4e2a-ab34-82d7b00e5265 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:02.507898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-10a8bfcb-c52b-43ce-8ef6-14b957f38316 req-4ecea3ed-ce61-4e2a-ab34-82d7b00e5265 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:02.508707 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-10a8bfcb-c52b-43ce-8ef6-14b957f38316 req-4ecea3ed-ce61-4e2a-ab34-82d7b00e5265 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:02.521336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-10a8bfcb-c52b-43ce-8ef6-14b957f38316 req-4ecea3ed-ce61-4e2a-ab34-82d7b00e5265 service nova] Creating event network-vif-plugged:c0379d91-6e4b-4a6c-895c-fc71a455dbb1 for instance 841a7957-ee73-4765-a850-406b1e720612 on hv-cinder-maste Oct 09 00:59:02.526065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-10a8bfcb-c52b-43ce-8ef6-14b957f38316 req-4ecea3ed-ce61-4e2a-ab34-82d7b00e5265 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033357 Oct 09 00:59:02.526670 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 212/421] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:59:02.650543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:02.658433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7e023c51-d6a8-432f-b95c-e5ccec381411 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.295828 Oct 09 00:59:02.658995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 210/422] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:02 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:03.161832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:03.162533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:03.169143 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:03.169629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:03.170067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:03.170756 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:03.171215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:03.171668 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:03.300099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:03.307651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2ef0db71-bf9f-4951-83e4-95ddc99c22f0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.150604 Oct 09 00:59:03.308359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 213/423] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:03 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:03.681324 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:03.681929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:03.688401 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:03.688879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:03.689903 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:03.690592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:03.691042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:03.691466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:03.818086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:03.825639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ccbbb5bb-a487-46c7-871f-122e38256d45 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.148212 Oct 09 00:59:03.826203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 211/424] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:03 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:04.330217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:04.330816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:04.336713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:04.337139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:04.337574 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:04.338253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:04.338702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:04.339122 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:04.474060 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:04.481900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a4595fb2-0351-47e8-a12e-8409dfebd116 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.155976 Oct 09 00:59:04.482474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 214/425] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:04 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:04.848479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:04.849082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:04.855488 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:04.856007 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:04.856490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:04.857119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:04.857607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:04.857997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:04.988197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:04.995838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b605814d-765c-47fc-bdf5-318dfa003a36 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.151787 Oct 09 00:59:04.996432 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 212/426] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:04 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:05.505330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:05.505940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:05.511959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:05.512444 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:05.512884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:05.513572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:05.514030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:05.514456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:05.645544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:05.653139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-841c4b19-a1bd-4f3f-9ee2-cad411b19120 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1265 microversion: 2.1 time: 0.152319 Oct 09 00:59:05.653693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 215/427] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:05 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:06.018284 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:06.018884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:06.025136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.025610 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.026036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:06.026706 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.027148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.027562 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:06.156623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:06.164165 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-59f75d11-22c9-4853-ba6d-cc654752c816 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.149810 Oct 09 00:59:06.165176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 213/428] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:06 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:06.489091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-dcd4dcf6-2bc6-454a-87ea-ed30efa9f4b5 req-7c0cf4bb-b162-4216-b680-de9aabf3a72d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f748f5a5-4a7a-48c7-8790-a5c7ed797db7", "name": "network-vif-plugged", "status": "completed", "tag": "d05ad078-2684-4d21-8255-efa3f61a8cb4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:06.499041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-dcd4dcf6-2bc6-454a-87ea-ed30efa9f4b5 req-7c0cf4bb-b162-4216-b680-de9aabf3a72d service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.499577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-dcd4dcf6-2bc6-454a-87ea-ed30efa9f4b5 req-7c0cf4bb-b162-4216-b680-de9aabf3a72d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.500065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-dcd4dcf6-2bc6-454a-87ea-ed30efa9f4b5 req-7c0cf4bb-b162-4216-b680-de9aabf3a72d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:06.512976 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-dcd4dcf6-2bc6-454a-87ea-ed30efa9f4b5 req-7c0cf4bb-b162-4216-b680-de9aabf3a72d service nova] Creating event network-vif-plugged:d05ad078-2684-4d21-8255-efa3f61a8cb4 for instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 on hv-cinder-maste Oct 09 00:59:06.518510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-dcd4dcf6-2bc6-454a-87ea-ed30efa9f4b5 req-7c0cf4bb-b162-4216-b680-de9aabf3a72d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033526 Oct 09 00:59:06.519297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 216/429] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:06.676563 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:06.677355 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:06.683796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.684496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.687461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:06.689642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.690302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.690906 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:06.877824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:06.885737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-533c3b1d-e570-4367-92ae-7b1c2eed76b7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1453 microversion: 2.1 time: 0.213881 Oct 09 00:59:06.886306 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 214/430] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:06 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1453 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:06.909802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:06.910822 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:06.917989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.918524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.919012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:06.919849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:06.920516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:06.921014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:07.059311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:07.067011 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e1809bc4-7003-41fa-bffe-75534c397d0a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1453 microversion: 2.1 time: 0.162170 Oct 09 00:59:07.067572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 217/431] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:06 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1453 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:07.186203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:07.186811 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:07.193059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:07.193545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:07.193980 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:07.194670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:07.195124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:07.195547 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:07.326414 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:07.334868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1addf10b-6838-45f0-8ea8-df894ee479c6 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.152464 Oct 09 00:59:07.335427 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 215/432] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:07 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:08.358422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:08.359658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:08.365696 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:08.366171 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:08.366646 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:08.367282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:08.367761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:08.368180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:08.557162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:08.567278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b1396809-a4e3-414e-92f6-bc1ad4007ecb tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1314 microversion: 2.1 time: 0.212542 Oct 09 00:59:08.568182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 218/433] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:08 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1314 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:09.089151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-fb923c8d-b4be-4cb5-b857-8a21343bb7fb req-c34f0094-0f72-4944-9f75-fbb820f23fec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "841a7957-ee73-4765-a850-406b1e720612", "tag": "c0379d91-6e4b-4a6c-895c-fc71a455dbb1"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:09.099164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fb923c8d-b4be-4cb5-b857-8a21343bb7fb req-c34f0094-0f72-4944-9f75-fbb820f23fec service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:09.099655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fb923c8d-b4be-4cb5-b857-8a21343bb7fb req-c34f0094-0f72-4944-9f75-fbb820f23fec service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:09.100083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-fb923c8d-b4be-4cb5-b857-8a21343bb7fb req-c34f0094-0f72-4944-9f75-fbb820f23fec service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:09.111468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-fb923c8d-b4be-4cb5-b857-8a21343bb7fb req-c34f0094-0f72-4944-9f75-fbb820f23fec service nova] Creating event network-changed:c0379d91-6e4b-4a6c-895c-fc71a455dbb1 for instance 841a7957-ee73-4765-a850-406b1e720612 on hv-cinder-maste Oct 09 00:59:09.116615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-fb923c8d-b4be-4cb5-b857-8a21343bb7fb req-c34f0094-0f72-4944-9f75-fbb820f23fec service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031298 Oct 09 00:59:09.117197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 216/434] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:09.590954 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:09.591613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:09.593729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=80 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:09.599952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:09.600445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:09.600883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:09.601587 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:09.602040 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:09.602468 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:09.731654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:09.739980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0088475f-f7f6-4316-a743-a91639dcdb6e tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1498 microversion: 2.1 time: 0.152797 Oct 09 00:59:09.740554 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 219/435] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:09 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1498 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:10.401545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:10.402127 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:10.411982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:10.412528 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:10.412967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:10.413673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:10.414329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:10.414852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:10.442188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 00:59:10.477623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:10.527300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3e6c67aa-7a3c-419e-8564-0c233eab1e6b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "DELETE /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 204 len: 0 microversion: 2.1 time: 0.133952 Oct 09 00:59:10.527901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 217/436] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 00:59:10 2022] DELETE /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 00:59:10.536782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:10.537339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:10.543552 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:10.544064 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:10.544502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:10.545187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:10.545804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:10.546375 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:10.675984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:10.683634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b411b20f-f963-4993-a64f-33017fcdee58 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1504 microversion: 2.1 time: 0.151345 Oct 09 00:59:10.684194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 220/437] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:10 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1504 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:11.706920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:11.707519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:11.714001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:11.714476 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:11.714904 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:11.715605 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:11.716045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:11.716459 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:11.846708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:11.854164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e74ca9f0-55ef-43c2-99b9-aea438f0d116 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1504 microversion: 2.1 time: 0.151591 Oct 09 00:59:11.854718 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 218/438] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:11 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1504 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:12.878501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:12.879108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:12.885821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:12.885821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:12.886131 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:12.886768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:12.887219 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:12.888420 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:13.022105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:13.030059 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-12f7ee7f-dd35-46c0-b62c-6eabb185f128 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 200 len: 1504 microversion: 2.1 time: 0.155913 Oct 09 00:59:13.030622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 221/439] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:12 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:13.158970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-1d69c366-cf5f-4ad3-a307-f7861846cf8c req-76ffbc1e-6e6d-423a-a670-f855ba1c3a20 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f748f5a5-4a7a-48c7-8790-a5c7ed797db7", "name": "network-vif-deleted", "tag": "d05ad078-2684-4d21-8255-efa3f61a8cb4"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:13.168796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1d69c366-cf5f-4ad3-a307-f7861846cf8c req-76ffbc1e-6e6d-423a-a670-f855ba1c3a20 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:13.169339 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1d69c366-cf5f-4ad3-a307-f7861846cf8c req-76ffbc1e-6e6d-423a-a670-f855ba1c3a20 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:13.169719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1d69c366-cf5f-4ad3-a307-f7861846cf8c req-76ffbc1e-6e6d-423a-a670-f855ba1c3a20 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:13.181883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-1d69c366-cf5f-4ad3-a307-f7861846cf8c req-76ffbc1e-6e6d-423a-a670-f855ba1c3a20 service nova] Creating event network-vif-deleted:d05ad078-2684-4d21-8255-efa3f61a8cb4 for instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 on hv-cinder-maste Oct 09 00:59:13.186784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-1d69c366-cf5f-4ad3-a307-f7861846cf8c req-76ffbc1e-6e6d-423a-a670-f855ba1c3a20 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031549 Oct 09 00:59:13.187642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 219/440] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:14.053707 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:14.054314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: f748f5a5-4a7a-48c7-8790-a5c7ed797db7] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:14.061080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:14.061564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:14.062002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:14.062690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:14.063142 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:14.063566 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:14.068200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:14.079766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] HTTP exception thrown: Instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 could not be found. Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Request method failure captured: Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 HTTP/1.1 Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhzGNKnb1XXZ071497MxXOB0umgJbpxQPfJrd0IR3N889eBnIPyyHsBUbxCqrQwn2yTZ3_U8HTspfcO1M_9tx9E1cUSN4X-qc7tqVlmKgniXhpgfpVNCu-VGLZY-hyZkorABpQfuDjEvZJkQtTRb8DSmGHJgQmAvJ6WQuohrryGjFHo Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 5edae5583d2e4a4688bef23f3a14eda4 Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesActionsTest-427535189 Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:59:14.081425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesActionsTest-427535189 Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 5edae5583d2e4a4688bef23f3a14eda4 Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesActionsTest-427535189 Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesActionsTest-427535189-project Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: adcbf206828d49b9bf2ae720f71068bc Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesActionsTest-427535189-project Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 could not be found. Oct 09 00:59:14.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'f748f5a5-4a7a-48c7-8790-a5c7ed797db7'} Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 could not be found. Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:59:14.084673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 could not be found. Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 could not be found. Oct 09 00:59:14.086617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:14.087946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Returning 404 to user: Instance f748f5a5-4a7a-48c7-8790-a5c7ed797db7 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:59:14.087946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-32def359-b325-459a-837f-ffd08a58de12 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7" status: 404 len: 111 microversion: 2.1 time: 0.033196 Oct 09 00:59:14.087946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 222/441] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:14 2022] GET /compute/v2.1/servers/f748f5a5-4a7a-48c7-8790-a5c7ed797db7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:59:14.238188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1246303514", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "networks": [{"uuid": "c27214a2-3f39-42ec-9d84-0333462c2425"}]}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:14.291889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] validate_networks() for [('c27214a2-3f39-42ec-9d84-0333462c2425', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:59:14.634607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:59:14.635028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:59:14.635496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:59:14.635937 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:59:14.636341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:59:14.645376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:14.645854 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:14.646280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:14.646989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:14.647436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:14.647892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:14.664911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Getting quotas for project 5edae5583d2e4a4688bef23f3a14eda4. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:59:14.668735 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Getting quotas for user adcbf206828d49b9bf2ae720f71068bc and project 5edae5583d2e4a4688bef23f3a14eda4. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:59:14.679194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:59:14.681541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:59:14.726903 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2b357a1a-6d3e-4570-80d2-1475be05dd97 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.493864 Oct 09 00:59:14.727794 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 220/442] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:59:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 495 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 00:59:14.738504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-37e56ac7-6ae5-406c-97f7-2804e1971f13 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:14.739120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-37e56ac7-6ae5-406c-97f7-2804e1971f13 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:14.792403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-37e56ac7-6ae5-406c-97f7-2804e1971f13 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:14.799180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-37e56ac7-6ae5-406c-97f7-2804e1971f13 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1214 microversion: 2.1 time: 0.064605 Oct 09 00:59:14.799749 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 223/443] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:14 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1214 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:15.822520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:15.823047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:15.829924 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:15.830410 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:15.830846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:15.831528 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:15.832003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:15.832431 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:15.892531 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:15.900234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-569d4eaf-9d84-4bcc-a92e-032fc4903ce8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1266 microversion: 2.1 time: 0.082140 Oct 09 00:59:15.900797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 221/444] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:15 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1266 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:16.922388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:16.923517 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:16.930110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:16.930605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:16.931044 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:16.931831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:16.932289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:16.932721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:17.073869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:17.082356 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-eb614a19-24e1-42a2-81a9-df629c512981 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.163739 Oct 09 00:59:17.083005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 224/445] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:16 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:17.643526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-79d1deb4-680a-46f1-b393-1c9b6ac0c8e2 req-d851cbe9-578f-4886-b0d2-4a41ef7a7538 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bade374f-83da-4ade-adaf-6f27f725883a", "tag": "4d6dd279-c688-428d-9c66-ec3efbf54fe7"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:17.655387 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-79d1deb4-680a-46f1-b393-1c9b6ac0c8e2 req-d851cbe9-578f-4886-b0d2-4a41ef7a7538 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:17.655933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-79d1deb4-680a-46f1-b393-1c9b6ac0c8e2 req-d851cbe9-578f-4886-b0d2-4a41ef7a7538 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:17.656477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-79d1deb4-680a-46f1-b393-1c9b6ac0c8e2 req-d851cbe9-578f-4886-b0d2-4a41ef7a7538 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:17.670027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-79d1deb4-680a-46f1-b393-1c9b6ac0c8e2 req-d851cbe9-578f-4886-b0d2-4a41ef7a7538 service nova] Creating event network-changed:4d6dd279-c688-428d-9c66-ec3efbf54fe7 for instance bade374f-83da-4ade-adaf-6f27f725883a on hv-cinder-maste Oct 09 00:59:17.674891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-79d1deb4-680a-46f1-b393-1c9b6ac0c8e2 req-d851cbe9-578f-4886-b0d2-4a41ef7a7538 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035587 Oct 09 00:59:17.675482 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 222/446] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:18.105439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:18.106049 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:18.111921 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:18.112401 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:18.112836 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:18.113518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:18.113972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:18.114394 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:18.245046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:18.253942 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2169c223-ab2b-4363-af87-a9d92f2fcee8 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.152418 Oct 09 00:59:18.254475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 225/447] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:18 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:19.277532 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:19.278249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:19.285653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:19.286133 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:19.286567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:19.287245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:19.287720 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:19.288160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:19.494005 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:19.502159 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0e0b19c7-b837-417e-a12e-5fa88e56b672 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.228792 Oct 09 00:59:19.502717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 223/448] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:19 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:20.525039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:20.525669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:20.532620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:20.533101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:20.533537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:20.534211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:20.534657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:20.535075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:20.664885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:20.672383 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-58027fd7-d0ed-4506-8bed-57e426ed4c8b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.151186 Oct 09 00:59:20.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 226/449] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:20 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:21.695493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:21.696176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:21.702803 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:21.703283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:21.703765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:21.705094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:21.705546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:21.705996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:21.835827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:21.843779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-17651474-bbfe-466f-9d01-540689d36ca9 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.152117 Oct 09 00:59:21.844331 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 224/450] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:21 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:22.051829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:22.052334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:22.058169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:22.058653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:22.059087 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:22.059794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:22.060275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:22.060696 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:22.183389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lazy-loading 'services' on Instance uuid 3247d2ef-8e30-4cee-b097-1957a0a3de81 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:59:22.303930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-52b1d417-e155-4e78-a11b-a67a79625c5d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-volume_attachments/adee1caf-ae25-496d-9f4c-b58d602ff8de" status: 202 len: 0 microversion: 2.1 time: 0.255874 Oct 09 00:59:22.304432 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 227/451] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 00:59:22 2022] DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-volume_attachments/adee1caf-ae25-496d-9f4c-b58d602ff8de => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:59:22.867313 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:22.867865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:22.874435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:22.874923 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:22.875354 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:22.876667 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:22.877117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:22.877548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:23.019533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:23.027739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-996d4c72-90cb-4e4b-9dd8-b74e4f2327bc tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.164256 Oct 09 00:59:23.028312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 225/452] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:22 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:24.051148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:24.051879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:24.058385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:24.058867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:24.059301 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:24.060022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:24.060477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:24.060906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:24.188576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:24.196320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c2e2bf96-bdd2-4b70-a9f3-d5b34e09852d tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.150266 Oct 09 00:59:24.197738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 228/453] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:24 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:24.480549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a2d1d821-337c-4a5e-a348-9a275a419cf9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:24.486870 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a2d1d821-337c-4a5e-a348-9a275a419cf9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:24.487341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a2d1d821-337c-4a5e-a348-9a275a419cf9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:24.487873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a2d1d821-337c-4a5e-a348-9a275a419cf9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:24.509270 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a2d1d821-337c-4a5e-a348-9a275a419cf9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Going to try to stop instance {{(pid=120633) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 09 00:59:24.554215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a2d1d821-337c-4a5e-a348-9a275a419cf9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action" status: 202 len: 0 microversion: 2.1 time: 0.077195 Oct 09 00:59:24.554806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 226/454] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:59:24 2022] POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:59:24.563027 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:24.563886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:24.569819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:24.570437 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:24.570936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:24.571782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:24.572244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:24.572673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:24.638020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-dc721717-487d-45a2-abd4-f95560fbe97f req-1979c11a-476b-4876-a323-610ddd407c3f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bade374f-83da-4ade-adaf-6f27f725883a", "name": "network-vif-plugged", "status": "completed", "tag": "4d6dd279-c688-428d-9c66-ec3efbf54fe7"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:24.648868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-dc721717-487d-45a2-abd4-f95560fbe97f req-1979c11a-476b-4876-a323-610ddd407c3f service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:24.649552 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-dc721717-487d-45a2-abd4-f95560fbe97f req-1979c11a-476b-4876-a323-610ddd407c3f service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:24.649980 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-dc721717-487d-45a2-abd4-f95560fbe97f req-1979c11a-476b-4876-a323-610ddd407c3f service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:24.662820 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:24.665047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-dc721717-487d-45a2-abd4-f95560fbe97f req-1979c11a-476b-4876-a323-610ddd407c3f service nova] Creating event network-vif-plugged:4d6dd279-c688-428d-9c66-ec3efbf54fe7 for instance bade374f-83da-4ade-adaf-6f27f725883a on hv-cinder-maste Oct 09 00:59:24.670046 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-dc721717-487d-45a2-abd4-f95560fbe97f req-1979c11a-476b-4876-a323-610ddd407c3f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035963 Oct 09 00:59:24.670634 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 227/455] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 00:59:24.714741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:24.722484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0ccdee5c-2210-40f6-851e-c87547259b71 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.163444 Oct 09 00:59:24.723039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 229/456] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:24 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:25.215733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:25.220192 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:25.220868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:25.227119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:25.227620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:25.228035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:25.228707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:25.229150 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:25.229568 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:25.364019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:25.372372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bc99ce11-a53a-4d13-9a7f-3dcf17f7b679 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.157750 Oct 09 00:59:25.373137 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 228/457] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:25 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:25.744396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:25.745028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:25.746970 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:25.753524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:25.754003 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:25.754440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:25.755147 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:25.755613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:25.756033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:25.886721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:25.894384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ba6645d2-5f78-411c-87cc-291bd49bceaf tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.153808 Oct 09 00:59:25.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 230/458] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:25 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:26.395394 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:26.396016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:26.402316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:26.402787 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:26.403214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:26.404240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:26.404688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:26.405104 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:26.533835 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:26.541192 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-83ee41b4-d162-4bb9-8832-604c98be1438 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1314 microversion: 2.1 time: 0.149581 Oct 09 00:59:26.541744 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 229/459] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:26 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:26.917708 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:26.918309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:26.924177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:26.924653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:26.925082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:26.925765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:26.926211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:26.926628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:27.056579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:27.065613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7b584055-7f73-46f6-ac45-7b66ac131a92 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.151574 Oct 09 00:59:27.066263 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 231/460] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:26 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:27.564883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:27.565593 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:27.571850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:27.572386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:27.572766 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:27.573449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:27.573916 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:27.574320 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:27.704452 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:27.712081 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-774964ad-47cf-438d-a94d-416b091254c1 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1498 microversion: 2.1 time: 0.151596 Oct 09 00:59:27.712653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 230/461] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:27 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1498 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:28.090152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:28.090761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:28.096826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:28.097308 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:28.097744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:28.098530 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:28.099066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:28.099492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:28.233233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:28.240811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-02e26d99-69c5-483b-8b5f-6d2c0ca46d5e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.154453 Oct 09 00:59:28.241368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 232/462] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:28 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:28.322853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:28.323343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:28.329906 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:28.330382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:28.330810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:28.331491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:28.331972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:28.332392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:28.356941 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 00:59:28.415185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:28.432789 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4291fc19-df58-408b-8ce4-2fe70ea3be77 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "DELETE /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 204 len: 0 microversion: 2.1 time: 0.113405 Oct 09 00:59:28.433310 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 231/463] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 00:59:28 2022] DELETE /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 00:59:28.443031 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:28.443654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:28.449728 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:28.450279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:28.450719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:28.451393 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:28.451866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:28.452310 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:28.584145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:28.591651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-507a43f4-1367-4a4b-98b2-a52f553c6eb0 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1504 microversion: 2.1 time: 0.152396 Oct 09 00:59:28.592199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 233/464] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:28 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1504 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:29.253289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:29.253864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:29.260164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.260643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.261075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.261765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.262213 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.262635 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.263516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:29.264184 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:29.270262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.270740 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.271203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.271917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.272372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.272796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.286236 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0978c42a-b011-4f61-9929-d843c36e9c67 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/ips" status: 200 len: 151 microversion: 2.1 time: 0.037094 Oct 09 00:59:29.286803 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 232/465] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 00:59:29 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/ips => generated 151 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:29.299476 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "abc87052-e51b-4588-a46f-73980b21e1e6"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:29.301768 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:29.307561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.308082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.308514 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.309196 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.309673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.310092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.389341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Invalid volume: volume abc87052-e51b-4588-a46f-73980b21e1e6 is already attached to instances: acc5e0e3-a218-4f87-b4cc-2efeb32e281e Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: POST /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/os-volume_attachments HTTP/1.1 Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 74 Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 00:59:29.391041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {"volumeAttachment": {"volumeId": "abc87052-e51b-4588-a46f-73980b21e1e6"}} Oct 09 00:59:29.392639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Invalid volume: volume abc87052-e51b-4588-a46f-73980b21e1e6 is already attached to instances: acc5e0e3-a218-4f87-b4cc-2efeb32e281e Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'server_id': '9988ceee-fd6b-43f1-ae12-74b98a9f128a', 'body': {'volumeAttachment': {'volumeId': 'abc87052-e51b-4588-a46f-73980b21e1e6'}}} Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume abc87052-e51b-4588-a46f-73980b21e1e6 is already attached to instances: acc5e0e3-a218-4f87-b4cc-2efeb32e281e Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:29.393581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume abc87052-e51b-4588-a46f-73980b21e1e6 is already attached to instances: acc5e0e3-a218-4f87-b4cc-2efeb32e281e Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:29.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume abc87052-e51b-4588-a46f-73980b21e1e6 is already attached to instances: acc5e0e3-a218-4f87-b4cc-2efeb32e281e Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 400 to user: Invalid volume: volume abc87052-e51b-4588-a46f-73980b21e1e6 is already attached to instances: acc5e0e3-a218-4f87-b4cc-2efeb32e281e {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-afcc8048-1754-4f0f-940f-2462bf1b0846 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.097391 Oct 09 00:59:29.397232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 233/466] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 00:59:29 2022] POST /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a/os-volume_attachments => generated 174 bytes in 102 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 09 00:59:29.400940 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:29.401426 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:29.407417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:29.410718 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.411227 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.411696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.412388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.412840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.413263 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.415454 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6c32c68a-b0f1-49f3-9fda-92f7d5dc51c7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.155736 Oct 09 00:59:29.416101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 234/467] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:29 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:29.417234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:29.438829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 00:59:29.509459 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-74444a98-3e4d-4a23-977a-7ef292528bd8 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 204 len: 0 microversion: 2.1 time: 0.111875 Oct 09 00:59:29.510032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 234/468] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 00:59:29 2022] DELETE /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 00:59:29.518283 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:29.518925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:29.524781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.525252 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.525680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.526348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.526869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.527281 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.615280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:29.616082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:29.621868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.622341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.622765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.623440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:29.623913 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:29.624333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:29.655524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:29.663896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-33af2ec7-9dbd-4b80-a1a1-b217a636fe09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1684 microversion: 2.1 time: 0.149539 Oct 09 00:59:29.664461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 235/469] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:29 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1684 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:29.751139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:29.758730 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5607f867-09c5-4241-bc7f-cf37bd6f3a21 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1504 microversion: 2.1 time: 0.147757 Oct 09 00:59:29.759286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 235/470] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:29 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1504 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:30.439137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:30.439778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:30.445809 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:30.446290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:30.446791 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:30.447473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:30.447948 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:30.448372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:30.579440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:30.587087 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1c0f760a-2506-41c6-bd31-eeba06a55dc3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.152453 Oct 09 00:59:30.587663 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 236/471] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:30 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:30.686794 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:30.687392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:30.694398 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:30.694818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:30.695246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:30.695990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:30.696440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:30.696855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:30.782671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:30.783372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:30.789261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:30.789745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:30.790238 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:30.790932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:30.791501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:30.791954 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:30.825719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:30.833449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-65d0c615-b40c-4fa0-be3a-eff5ec5d4c86 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1684 microversion: 2.1 time: 0.150433 Oct 09 00:59:30.834211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 236/472] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:30 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1684 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:30.922930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:30.930592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6777e87c-960f-49ff-bdbd-954f97110666 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1504 microversion: 2.1 time: 0.152519 Oct 09 00:59:30.931150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 237/473] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:30 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1504 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:31.609770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:31.610391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:31.616372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:31.616853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:31.617285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:31.617961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:31.618405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:31.618823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:31.743400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:31.750908 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-34a19e8f-93be-4d3a-8480-decde9abdf51 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.145032 Oct 09 00:59:31.752017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 237/474] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:31 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:31.857484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:31.858082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:31.865206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:31.865633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:31.866075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:31.866772 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:31.867228 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:31.867691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:31.953515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:31.954239 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:31.960157 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:31.960727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:31.961225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:31.961986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:31.962526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:31.962955 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:31.994227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:32.002762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-00af378d-0998-40cf-8d05-bdf2f46076f6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1684 microversion: 2.1 time: 0.149140 Oct 09 00:59:32.003391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 238/475] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:31 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1684 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:32.216173 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:32.223859 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7bc1dcf9-e73a-40af-8a40-3f06bf76b737 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1504 microversion: 2.1 time: 0.274208 Oct 09 00:59:32.224422 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 238/476] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:31 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1504 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:32.775336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:32.776110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:32.782473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:32.782956 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:32.783408 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:32.784114 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:32.784568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:32.784989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:32.918596 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:32.922102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:32.928217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8832134a-1e00-4050-87ad-47d129ae290a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.156698 Oct 09 00:59:32.928766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 239/477] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:32 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:32.957801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-4293ab10-4eac-434c-ad7c-b8450971cfd8 req-569cc53f-319c-4b3c-b767-847a837a5882 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bade374f-83da-4ade-adaf-6f27f725883a", "name": "network-vif-deleted", "tag": "4d6dd279-c688-428d-9c66-ec3efbf54fe7"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:32.968775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-4293ab10-4eac-434c-ad7c-b8450971cfd8 req-569cc53f-319c-4b3c-b767-847a837a5882 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:32.969252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-4293ab10-4eac-434c-ad7c-b8450971cfd8 req-569cc53f-319c-4b3c-b767-847a837a5882 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:32.970178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-4293ab10-4eac-434c-ad7c-b8450971cfd8 req-569cc53f-319c-4b3c-b767-847a837a5882 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:32.982875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-4293ab10-4eac-434c-ad7c-b8450971cfd8 req-569cc53f-319c-4b3c-b767-847a837a5882 service nova] Creating event network-vif-deleted:4d6dd279-c688-428d-9c66-ec3efbf54fe7 for instance bade374f-83da-4ade-adaf-6f27f725883a on hv-cinder-maste Oct 09 00:59:32.989493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-4293ab10-4eac-434c-ad7c-b8450971cfd8 req-569cc53f-319c-4b3c-b767-847a837a5882 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037276 Oct 09 00:59:32.990032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 239/478] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:33.026419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:33.027146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:33.036847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.037493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.038145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:33.039073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.039785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.040545 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:33.109801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:33.117812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-756cccde-a1cb-4b55-a666-8d3b4a39cebf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1616 microversion: 2.1 time: 0.095318 Oct 09 00:59:33.118368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 240/479] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:33 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1616 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:33.247048 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:33.247861 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:33.254765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.255253 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.255719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:33.256420 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.256893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.257312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:33.318936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.320208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.320881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:33.321832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lazy-loading 'fault' on Instance uuid bade374f-83da-4ade-adaf-6f27f725883a {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:59:33.378094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'adcbf206828d49b9bf2ae720f71068bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5edae5583d2e4a4688bef23f3a14eda4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:33.389332 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3d8e0d19-8451-4d53-aebd-2eeec455e323 tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 200 len: 1304 microversion: 2.1 time: 0.146565 Oct 09 00:59:33.389915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 240/480] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:33 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 1304 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:33.950700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:33.951433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:33.966160 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.967237 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.967899 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:33.968912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:33.969447 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:33.969922 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:34.143792 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:34.145862 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:34.145994 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:34.154225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-13640cf0-fa1d-41d4-80aa-1c7532ea46e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1677 microversion: 2.1 time: 0.207286 Oct 09 00:59:34.154792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 241/481] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:33 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1677 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:34.155464 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:34.156229 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:34.156763 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:34.157615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:34.158165 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:34.158690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:34.201620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:34.202506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:34.203139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:34.203746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'fault' on Instance uuid 9988ceee-fd6b-43f1-ae12-74b98a9f128a {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:59:34.249983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:34.261075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-087c368e-41a6-4b49-8fff-f51537f10127 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 200 len: 1332 microversion: 2.1 time: 0.122120 Oct 09 00:59:34.261651 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 241/482] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:34 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 1332 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:34.414410 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:34.414947 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] [instance: bade374f-83da-4ade-adaf-6f27f725883a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:34.420990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:34.421473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:34.421906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:34.423339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:34.423981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:34.424567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:34.438512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] HTTP exception thrown: Instance bade374f-83da-4ade-adaf-6f27f725883a could not be found. Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Request method failure captured: Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a HTTP/1.1 Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhzGNKnb1XXZ071497MxXOB0umgJbpxQPfJrd0IR3N889eBnIPyyHsBUbxCqrQwn2yTZ3_U8HTspfcO1M_9tx9E1cUSN4X-qc7tqVlmKgniXhpgfpVNCu-VGLZY-hyZkorABpQfuDjEvZJkQtTRb8DSmGHJgQmAvJ6WQuohrryGjFHo Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 5edae5583d2e4a4688bef23f3a14eda4 Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesActionsTest-427535189 Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 00:59:34.440255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5edae5583d2e4a4688bef23f3a14eda4"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesActionsTest-427535189 Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 5edae5583d2e4a4688bef23f3a14eda4 Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesActionsTest-427535189 Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesActionsTest-427535189-project Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: adcbf206828d49b9bf2ae720f71068bc Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesActionsTest-427535189-project Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance bade374f-83da-4ade-adaf-6f27f725883a could not be found. Oct 09 00:59:34.441761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'bade374f-83da-4ade-adaf-6f27f725883a'} Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bade374f-83da-4ade-adaf-6f27f725883a could not be found. Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:59:34.442717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bade374f-83da-4ade-adaf-6f27f725883a could not be found. Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bade374f-83da-4ade-adaf-6f27f725883a could not be found. Oct 09 00:59:34.444146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 00:59:34.445418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] Returning 404 to user: Instance bade374f-83da-4ade-adaf-6f27f725883a could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:59:34.445418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c109ecb7-4eed-4013-8190-072b48d06c7b tempest-VolumesActionsTest-427535189 tempest-VolumesActionsTest-427535189-project] 10.222.0.33 "GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a" status: 404 len: 111 microversion: 2.1 time: 0.031347 Oct 09 00:59:34.445418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 242/483] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:34 2022] GET /compute/v2.1/servers/bade374f-83da-4ade-adaf-6f27f725883a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:59:35.006709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-3362b738-efef-4d98-b611-b32fb0736412 req-1e612881-2ac5-46d0-b92a-d5feb09904c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9988ceee-fd6b-43f1-ae12-74b98a9f128a", "name": "network-vif-deleted", "tag": "9f833987-35a7-48c2-8720-3ab67f5287e5"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:35.017695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-3362b738-efef-4d98-b611-b32fb0736412 req-1e612881-2ac5-46d0-b92a-d5feb09904c0 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.018182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-3362b738-efef-4d98-b611-b32fb0736412 req-1e612881-2ac5-46d0-b92a-d5feb09904c0 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.018615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-3362b738-efef-4d98-b611-b32fb0736412 req-1e612881-2ac5-46d0-b92a-d5feb09904c0 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.030882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-3362b738-efef-4d98-b611-b32fb0736412 req-1e612881-2ac5-46d0-b92a-d5feb09904c0 service nova] Creating event network-vif-deleted:9f833987-35a7-48c2-8720-3ab67f5287e5 for instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a on hv-cinder-maste Oct 09 00:59:35.037616 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-3362b738-efef-4d98-b611-b32fb0736412 req-1e612881-2ac5-46d0-b92a-d5feb09904c0 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034848 Oct 09 00:59:35.038205 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 242/484] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:35.178199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:35.178812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:35.184857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.185340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.185778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.186461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.186916 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.187360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.285728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:35.286719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:35.294173 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.294735 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.295276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.296354 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.297182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.298511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.319656 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a HTTP/1.1 Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 00:59:35.321617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 00:59:35.323045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '9988ceee-fd6b-43f1-ae12-74b98a9f128a'} Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:59:35.323925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 00:59:35.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:35.327302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:59:35.328186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9367d4f0-a0f6-4910-8aab-26452a861ca7 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 404 len: 111 microversion: 2.1 time: 0.047686 Oct 09 00:59:35.328755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 243/485] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:35 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:59:35.333661 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:35.336871 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1a79cd9d-9055-48d0-97d3-c046211135d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '.version_select at 0x7f7268a95ca0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:35.343057 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-daed817c-292d-4df1-b32e-a2594a9f847e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1654 microversion: 2.1 time: 0.168685 Oct 09 00:59:35.347654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 243/486] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:35 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1654 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:35.364795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1a79cd9d-9055-48d0-97d3-c046211135d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1986031973" status: 202 len: 0 microversion: 2.1 time: 0.031446 Oct 09 00:59:35.365482 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 244/487] 10.222.0.33 () {58 vars in 1221 bytes} [Sun Oct 9 00:59:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1986031973 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 00:59:35.369043 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b1af0823-f8ac-45a6-bed4-1671e5765a99 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:35.375167 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1af0823-f8ac-45a6-bed4-1671e5765a99 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.375712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1af0823-f8ac-45a6-bed4-1671e5765a99 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.376186 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1af0823-f8ac-45a6-bed4-1671e5765a99 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.398881 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b1af0823-f8ac-45a6-bed4-1671e5765a99 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Going to try to start instance {{(pid=120632) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 09 00:59:35.448823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b1af0823-f8ac-45a6-bed4-1671e5765a99 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action" status: 202 len: 0 microversion: 2.1 time: 0.083442 Oct 09 00:59:35.451753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 244/488] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 00:59:35 2022] POST /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 00:59:35.459098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:35.459808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:35.470053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.471108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.471972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.472847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:35.473381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:35.473887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:35.629139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:35.637198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4f749c7f-e7b2-48ce-babb-b9435cfe701e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1663 microversion: 2.1 time: 0.182941 Oct 09 00:59:35.637756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 245/489] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:35 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1663 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:36.073721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:36.074377 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:36.080857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:36.081409 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:36.081843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:36.082561 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:36.083436 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:36.083436 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:36.164250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'services' on Instance uuid acc5e0e3-a218-4f87-b4cc-2efeb32e281e {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:59:36.234877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-adaca1be-a133-425f-b6ca-266f7b03ec3b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments/abc87052-e51b-4588-a46f-73980b21e1e6" status: 202 len: 0 microversion: 2.1 time: 0.165216 Oct 09 00:59:36.236094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 245/490] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 00:59:36 2022] DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments/abc87052-e51b-4588-a46f-73980b21e1e6 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:59:36.244379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:36.246527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:36.252385 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:36.252867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:36.253304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:36.253983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:36.254434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:36.254857 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:36.284711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-558624ec-0d0f-4f5d-9118-a4ee7232a761 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044243 Oct 09 00:59:36.285248 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 246/491] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 00:59:36 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:36.661688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:36.662389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:36.668867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:36.669353 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:36.669790 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:36.670473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:36.670929 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:36.671352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:36.796684 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:36.804365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a7e3e6ff-2f5d-4539-9a30-89d41e5dfd42 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1663 microversion: 2.1 time: 0.147456 Oct 09 00:59:36.804929 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 246/492] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:36 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1663 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:37.297212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:37.299918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:37.306151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:37.306644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:37.307079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:37.307782 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:37.308228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:37.308641 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:37.337709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-21b52b72-72bd-43d5-80e5-6706b24b2ec6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044398 Oct 09 00:59:37.338254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 247/493] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 00:59:37 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:37.827787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:37.828390 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:37.834837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:37.835315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:37.835802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:37.836486 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:37.836943 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:37.837366 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:37.966102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:37.974403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-abf151c0-cf5d-4095-ba63-f68188c2094e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1663 microversion: 2.1 time: 0.150534 Oct 09 00:59:37.974969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 247/494] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:37 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1663 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:38.350194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:38.352267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:38.358686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:38.359169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:38.359620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:38.360311 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:38.360760 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:38.361181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:38.390363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-acbfb605-0809-410d-8599-ad5adf609daf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044220 Oct 09 00:59:38.390897 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 248/495] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 00:59:38 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:38.997148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:38.997785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:39.004174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.004656 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.005097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.005785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.006240 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.006675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.137017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:39.146137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ef14929c-f63b-486e-906d-6a5397f64410 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1663 microversion: 2.1 time: 0.152813 Oct 09 00:59:39.146705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 248/496] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:38 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1663 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:39.402604 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:39.405038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:39.411458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.412155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.412588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.413329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.413938 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.414410 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.445813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9a29eb61-246b-435b-be04-c8100e5efebb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047359 Oct 09 00:59:39.446672 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 249/497] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 00:59:39 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 00:59:39.511615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:39.512124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:39.518267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.518735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.519173 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.520007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.520385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.520738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.552561 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 00:59:39.644529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=64 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:39.663426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d5c8a039-7b74-4a5b-9a20-185806857794 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 204 len: 0 microversion: 2.1 time: 0.154995 Oct 09 00:59:39.664459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 249/498] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 00:59:39 2022] DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 00:59:39.672416 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:39.674360 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:39.680713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.681467 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.682121 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.683034 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:39.684308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:39.684953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:39.832481 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:39.841520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e310bf73-8cc6-4e95-ac01-22440b7e1993 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1682 microversion: 2.1 time: 0.172202 Oct 09 00:59:39.842477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 250/499] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:39 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1682 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:40.169933 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:40.171275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:40.180918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:40.181417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:40.181861 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:40.182562 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:40.189115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:40.189724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:40.345018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:40.353662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-46bf3bfc-0192-4d32-97ce-34787cbb016b tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1667 microversion: 2.1 time: 0.187865 Oct 09 00:59:40.354351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 250/500] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:40 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1667 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:40.866370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:40.867138 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:40.873566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:40.874433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:40.875111 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:40.876402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:40.877227 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:40.878060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:40.902365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:41.290832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:41.308843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2231e4c0-9372-4b6f-8e9d-9924fc88992d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1682 microversion: 2.1 time: 0.445572 Oct 09 00:59:41.310491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 251/501] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:40 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1682 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:42.338327 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:42.340032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:42.346929 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:42.347487 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:42.348040 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:42.348878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:42.349411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:42.349829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:42.531776 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:42.549952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4a5b6e0b-f7ba-4682-bc6d-b3c047fd21c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1682 microversion: 2.1 time: 0.215306 Oct 09 00:59:42.573032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 251/502] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:42 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1682 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:43.421566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-ea75cc5c-04b8-4510-ad7f-78f2d3599e7e req-c7353be9-3e76-46ff-98c1-0d7b520130c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "acc5e0e3-a218-4f87-b4cc-2efeb32e281e", "name": "network-vif-deleted", "tag": "033058db-9b72-4aa7-b12d-bf2294dc0aa3"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:43.435412 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ea75cc5c-04b8-4510-ad7f-78f2d3599e7e req-c7353be9-3e76-46ff-98c1-0d7b520130c9 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:43.436119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ea75cc5c-04b8-4510-ad7f-78f2d3599e7e req-c7353be9-3e76-46ff-98c1-0d7b520130c9 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:43.436731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ea75cc5c-04b8-4510-ad7f-78f2d3599e7e req-c7353be9-3e76-46ff-98c1-0d7b520130c9 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:43.454382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-ea75cc5c-04b8-4510-ad7f-78f2d3599e7e req-c7353be9-3e76-46ff-98c1-0d7b520130c9 service nova] Creating event network-vif-deleted:033058db-9b72-4aa7-b12d-bf2294dc0aa3 for instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e on hv-cinder-maste Oct 09 00:59:43.461886 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-ea75cc5c-04b8-4510-ad7f-78f2d3599e7e req-c7353be9-3e76-46ff-98c1-0d7b520130c9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043778 Oct 09 00:59:43.462712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 252/503] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:43.580492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:43.581236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:43.590854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:43.591346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:43.591815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:43.592518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:43.599005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:43.599996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:43.667960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:43.668725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:43.669321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:43.669963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'fault' on Instance uuid acc5e0e3-a218-4f87-b4cc-2efeb32e281e {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 00:59:43.727881 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:43.740833 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3e67a1bf-03e6-4258-b384-e0f62ff11897 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 200 len: 1332 microversion: 2.1 time: 0.164303 Oct 09 00:59:43.744003 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 252/504] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:43 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 1332 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:44.767326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:44.767868 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:44.780183 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:44.780764 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:44.781189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:44.785268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:44.785954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:44.786527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:44.802434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e HTTP/1.1 Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 00:59:44.804119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 00:59:44.805731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'acc5e0e3-a218-4f87-b4cc-2efeb32e281e'} Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 00:59:44.806629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 00:59:44.808337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 00:59:44.809943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 00:59:44.813126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-25d56cbf-41e1-4cca-bb51-9f1d7230b985 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 404 len: 111 microversion: 2.1 time: 0.048201 Oct 09 00:59:44.813740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 253/505] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:44 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 00:59:44.822384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-20cfb22f-d153-4f9b-8729-4421bd366dc2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '.version_select at 0x7f7268afb820>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:44.852567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-20cfb22f-d153-4f9b-8729-4421bd366dc2 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2130470652" status: 202 len: 0 microversion: 2.1 time: 0.033111 Oct 09 00:59:44.855277 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 253/506] 10.222.0.33 () {58 vars in 1221 bytes} [Sun Oct 9 00:59:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2130470652 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 00:59:45.616652 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-543bae9e-6ab5-4315-bb13-095bcf1307c1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: .version_select at 0x7f7268a7ba60>, body: {"keypair": {"name": "tempest-keypair-2048709389", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFSjESYBqOdok/iPU9AUCrbuv+2dh2E6xIcJTYUzfxxx4mWuf8tlSmSrlMitDa8ist5FOgti0d1+w/6KqQYqSlITeCax5vEW+CvwpZo9wWnOFpM/v4TaLJJAu/QWAllqLQ=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:45.634996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-543bae9e-6ab5-4315-bb13-095bcf1307c1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:59:45.639038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-543bae9e-6ab5-4315-bb13-095bcf1307c1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:59:45.663841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-543bae9e-6ab5-4315-bb13-095bcf1307c1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.050956 Oct 09 00:59:45.664432 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 254/507] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 00:59:45 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:47.169467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1154547020", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ef6b019b-3ff0-42c3-b2bd-354f2c856ccc"}], "security_groups": [{"name": "tempest-securitygroup--238333962"}], "key_name": "tempest-keypair-2048709389", "user_data": "}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:47.299688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] validate_networks() for [('ef6b019b-3ff0-42c3-b2bd-354f2c856ccc', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 00:59:47.735381 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 00:59:47.736829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 00:59:47.737755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 00:59:47.738513 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 00:59:47.739147 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 00:59:47.754712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:47.755744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:47.756358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:47.757445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:47.758080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:47.758665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:47.778678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 00:59:47.783512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 00:59:47.795450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 00:59:47.798433 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 00:59:47.870891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5a571609-30d8-4927-8262-f26ae8e6ae45 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.705496 Oct 09 00:59:47.871939 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 254/508] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 00:59:47 2022] POST /compute/v2.1/servers => generated 401 bytes in 707 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 09 00:59:47.885616 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-71e86e0f-0a0e-4fbb-91ab-3ba1d4c88462 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:47.887216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-71e86e0f-0a0e-4fbb-91ab-3ba1d4c88462 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:47.965191 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-71e86e0f-0a0e-4fbb-91ab-3ba1d4c88462 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:47.976037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-71e86e0f-0a0e-4fbb-91ab-3ba1d4c88462 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1242 microversion: 2.1 time: 0.095330 Oct 09 00:59:47.977335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 255/509] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:47 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1242 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:48.998622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:48.999911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:49.007293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:49.007742 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:49.008174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:49.008889 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:49.009316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:49.009915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:49.013800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:49.086956 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:49.095760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e7d09b3d-f2c5-4f30-aa44-6a1b9f5f0836 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1294 microversion: 2.1 time: 0.101043 Oct 09 00:59:49.096439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 255/510] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:48 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1294 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:50.124455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:50.125783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:50.132546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:50.133416 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:50.134103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:50.135478 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:50.136223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:50.136871 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:50.267538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:50.275966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-83c6c1c3-b89a-4152-844d-8836dbccb815 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1300 microversion: 2.1 time: 0.155313 Oct 09 00:59:50.296515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 256/511] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:50 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1300 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 00:59:51.304547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:51.305340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:51.323854 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-f79747c9-f0ac-4a36-95c8-43a0e5942fd2 req-4e2dafc0-8f11-4162-baae-4bc0ae51115d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "188a0b8d-23e2-4769-94bf-85277c3b0b66", "tag": "29a59039-ac16-488f-ad9d-1a896696e3f6"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:51.333040 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:51.333649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:51.334157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:51.337044 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:51.337599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:51.338142 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:51.343077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-f79747c9-f0ac-4a36-95c8-43a0e5942fd2 req-4e2dafc0-8f11-4162-baae-4bc0ae51115d service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:51.343683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-f79747c9-f0ac-4a36-95c8-43a0e5942fd2 req-4e2dafc0-8f11-4162-baae-4bc0ae51115d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:51.347087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-f79747c9-f0ac-4a36-95c8-43a0e5942fd2 req-4e2dafc0-8f11-4162-baae-4bc0ae51115d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:51.374214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-f79747c9-f0ac-4a36-95c8-43a0e5942fd2 req-4e2dafc0-8f11-4162-baae-4bc0ae51115d service nova] Creating event network-changed:29a59039-ac16-488f-ad9d-1a896696e3f6 for instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 on hv-cinder-maste Oct 09 00:59:51.383521 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-f79747c9-f0ac-4a36-95c8-43a0e5942fd2 req-4e2dafc0-8f11-4162-baae-4bc0ae51115d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068324 Oct 09 00:59:51.393266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 257/512] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:51.529323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:51.541312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-28ea0874-5b3a-4c48-9d3f-6081a7233e15 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1367 microversion: 2.1 time: 0.240974 Oct 09 00:59:51.552466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 256/513] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:51 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1367 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:52.569051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:52.570467 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:52.577292 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:52.578145 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:52.578818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:52.579791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:52.580334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:52.581314 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:52.724852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:52.733282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3c2d404f-7ff6-45a5-94e4-6ffea1bbdf80 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1367 microversion: 2.1 time: 0.168264 Oct 09 00:59:52.734060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 258/514] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:52 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1367 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:53.756158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:53.756763 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:53.762986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:53.763474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:53.764008 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:53.764712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:53.765167 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:53.766111 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:53.954156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:53.962727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2f18b204-9a8f-4570-b19a-a2c4d9b05fea tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1367 microversion: 2.1 time: 0.210435 Oct 09 00:59:53.963511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 257/515] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:53 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1367 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:54.985646 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:54.986236 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:54.992647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:54.993129 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:54.993565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:54.994256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:54.994709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:54.995133 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:55.124307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:55.131861 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-348306ad-8fc0-463a-b9ec-ae8b88961041 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1367 microversion: 2.1 time: 0.150025 Oct 09 00:59:55.132412 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 259/516] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:54 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:55.433030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-264ac7a7-bfe1-49f5-a7bf-6b77eccbfba0 req-2cf5a62d-786e-4c46-9b37-5abc0694e331 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "188a0b8d-23e2-4769-94bf-85277c3b0b66", "name": "network-vif-plugged", "status": "completed", "tag": "29a59039-ac16-488f-ad9d-1a896696e3f6"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:55.443102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-264ac7a7-bfe1-49f5-a7bf-6b77eccbfba0 req-2cf5a62d-786e-4c46-9b37-5abc0694e331 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:55.443580 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-264ac7a7-bfe1-49f5-a7bf-6b77eccbfba0 req-2cf5a62d-786e-4c46-9b37-5abc0694e331 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:55.444041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-264ac7a7-bfe1-49f5-a7bf-6b77eccbfba0 req-2cf5a62d-786e-4c46-9b37-5abc0694e331 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:55.455792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-264ac7a7-bfe1-49f5-a7bf-6b77eccbfba0 req-2cf5a62d-786e-4c46-9b37-5abc0694e331 service nova] Creating event network-vif-plugged:29a59039-ac16-488f-ad9d-1a896696e3f6 for instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 on hv-cinder-maste Oct 09 00:59:55.460820 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-264ac7a7-bfe1-49f5-a7bf-6b77eccbfba0 req-2cf5a62d-786e-4c46-9b37-5abc0694e331 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031468 Oct 09 00:59:55.461413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 258/517] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:56.154797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:56.155385 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:56.162215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:56.162794 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:56.163112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:56.163911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:56.164267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:56.164692 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:56.306294 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:56.314010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-84296969-57e0-4ee8-a91d-133f2cf672c9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1367 microversion: 2.1 time: 0.163645 Oct 09 00:59:56.314676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 260/518] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:56 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1367 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:56.617159 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "af3f6b36-1fc1-4dde-9076-c1ec90e38994"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:56.619618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:56.625705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:56.626213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:56.626644 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:56.627321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:56.627798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:56.628219 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:56.943637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-41a07764-7d6b-4185-990c-efbbbd00b4f8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.330535 Oct 09 00:59:56.944354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 259/519] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 00:59:56 2022] POST /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments => generated 194 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:57.337208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:57.337726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:57.344194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:57.344703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:57.345137 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:57.345823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:57.346271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:57.346691 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:57.470888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:57.478810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b54e8f23-b244-4827-b110-ebba8b119da6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1367 microversion: 2.1 time: 0.145468 Oct 09 00:59:57.479363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 261/520] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:57 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1367 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:58.500169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:58.501075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:58.505656 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:58.506360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:58.513356 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:58.513839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:58.514274 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:58.514962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:58.515413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:58.515864 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:58.651320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 00:59:58.659084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c787d352-1373-4cc7-8cc6-a8a5e5e0f83d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1558 microversion: 2.1 time: 0.160436 Oct 09 00:59:58.659698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 260/521] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 00:59:58 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1558 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 00:59:58.681285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 00:59:58.681846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 00:59:58.683865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:58.684737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=86,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 00:59:58.691419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:58.691939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:58.692369 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:58.693054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:58.693500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:58.693922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:58.774876 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3ec7a8fb-a11c-45d5-bc22-d9b0c7745240 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.097373 Oct 09 00:59:58.775582 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 262/522] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 00:59:58 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 00:59:59.744478 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-74f3e027-4027-4bda-9bc5-7469cbee7496 req-49dca5ce-4faf-4b24-aca4-a9a3713a31d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "188a0b8d-23e2-4769-94bf-85277c3b0b66", "tag": "29a59039-ac16-488f-ad9d-1a896696e3f6"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 00:59:59.754408 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-74f3e027-4027-4bda-9bc5-7469cbee7496 req-49dca5ce-4faf-4b24-aca4-a9a3713a31d8 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 00:59:59.754885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-74f3e027-4027-4bda-9bc5-7469cbee7496 req-49dca5ce-4faf-4b24-aca4-a9a3713a31d8 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 00:59:59.755320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-74f3e027-4027-4bda-9bc5-7469cbee7496 req-49dca5ce-4faf-4b24-aca4-a9a3713a31d8 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 00:59:59.766235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-74f3e027-4027-4bda-9bc5-7469cbee7496 req-49dca5ce-4faf-4b24-aca4-a9a3713a31d8 service nova] Creating event network-changed:29a59039-ac16-488f-ad9d-1a896696e3f6 for instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 on hv-cinder-maste Oct 09 00:59:59.772089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-74f3e027-4027-4bda-9bc5-7469cbee7496 req-49dca5ce-4faf-4b24-aca4-a9a3713a31d8 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031155 Oct 09 00:59:59.772751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 261/523] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 00:59:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:04.767650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-456877377", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:04.819173 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] validate_networks() for None {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:00:05.086338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:00:05.086758 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:00:05.087224 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:00:05.087647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:00:05.088059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:00:05.096947 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:05.097420 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:05.097846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:05.099063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:05.099509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:05.099951 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:05.116349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Getting quotas for project 9a6e25abae6b4b499a402bc4585bd06f. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:00:05.120185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Getting quotas for user 5becb8eb7b8d4c619f8164de3389a1ce and project 9a6e25abae6b4b499a402bc4585bd06f. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:00:05.130980 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:00:05.133228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:00:05.179338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-79aa2e82-8d48-4957-b4b6-25c52d8b02f4 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.416154 Oct 09 01:00:05.180030 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 263/524] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:00:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 418 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:00:05.192134 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b2dba1fe-63c0-4f99-b666-089ac68c8164 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:05.193404 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b2dba1fe-63c0-4f99-b666-089ac68c8164 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:05.247096 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b2dba1fe-63c0-4f99-b666-089ac68c8164 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:05.254146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b2dba1fe-63c0-4f99-b666-089ac68c8164 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1214 microversion: 2.1 time: 0.066805 Oct 09 01:00:05.254702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 262/525] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:05 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1214 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:05.559561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:05.561881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:05.568857 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:05.569215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:05.569653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:05.570349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:05.570799 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:05.571240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:05.593895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lazy-loading 'flavor' on Instance uuid 50224067-d3e0-4567-bc63-31ace3a211c6 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:00:05.720340 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1dc7f3eb-a3c8-4f27-99bd-adc450c4c220 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/action" status: 200 len: 32 microversion: 2.1 time: 0.164822 Oct 09 01:00:05.721043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 264/526] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 01:00:05 2022] POST /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/action => generated 32 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:00:05.731276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:05.731914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:05.738202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:05.738685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:05.739783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:05.740498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:05.740949 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:05.741496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:05.888279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:05.896798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d5266853-caf0-4190-ace8-6b90311a1920 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.169353 Oct 09 01:00:05.897675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 263/527] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:05 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:00:06.277844 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:06.278549 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:06.284662 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:06.285083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:06.285520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:06.286756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:06.287208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:06.287650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:06.360266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:06.367888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-76e86850-e12e-4cbc-950b-2f77d13bffc0 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1266 microversion: 2.1 time: 0.094972 Oct 09 01:00:06.368455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 265/528] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:06 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1266 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:06.923677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:06.924316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:06.931159 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:06.931676 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:06.932114 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:06.932813 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:06.933265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:06.933779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:07.073160 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:07.081736 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d7ef0314-194d-4779-b8d4-5abc99f5a61a tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.161898 Oct 09 01:00:07.082771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 264/529] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:06 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:07.390359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:07.390977 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:07.400190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:07.400757 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:07.401669 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:07.401898 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:07.402660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:07.403089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:07.599579 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:07.607566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-90e1e85a-f406-4b3c-973b-0abb018bfae6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.221056 Oct 09 01:00:07.608319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 266/530] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:07 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:08.105532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:08.106150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:08.112303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:08.112789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:08.113229 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:08.113951 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:08.114407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:08.114832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:08.254079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-c94a822b-0965-4f06-96fe-495f6158e3cd req-94e1cbc9-119f-4b52-a0a7-1eea3ba20c5d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a6b016ec-cbc1-4ad4-8e80-e79fddc5360d", "tag": "48bbfa7d-9313-405d-883a-e594cf9f5314"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:08.262180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:08.272666 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-c94a822b-0965-4f06-96fe-495f6158e3cd req-94e1cbc9-119f-4b52-a0a7-1eea3ba20c5d service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:08.276017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-c94a822b-0965-4f06-96fe-495f6158e3cd req-94e1cbc9-119f-4b52-a0a7-1eea3ba20c5d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:08.276455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-c94a822b-0965-4f06-96fe-495f6158e3cd req-94e1cbc9-119f-4b52-a0a7-1eea3ba20c5d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:08.279264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f44b9452-1076-4df9-b1c2-ba25c7fc9557 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.177532 Oct 09 01:00:08.279888 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 265/531] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:08 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:08.298211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-c94a822b-0965-4f06-96fe-495f6158e3cd req-94e1cbc9-119f-4b52-a0a7-1eea3ba20c5d service nova] Creating event network-changed:48bbfa7d-9313-405d-883a-e594cf9f5314 for instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d on hv-cinder-maste Oct 09 01:00:08.303042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-c94a822b-0965-4f06-96fe-495f6158e3cd req-94e1cbc9-119f-4b52-a0a7-1eea3ba20c5d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052691 Oct 09 01:00:08.304232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 267/532] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:08.632037 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:08.632676 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:08.638634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:08.639112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:08.639550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:08.640277 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:08.640729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:08.641169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:08.769833 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:08.777792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bd12c2f5-0af6-4a52-85b6-3d4674502a30 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.149989 Oct 09 01:00:08.778352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 266/533] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:08 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:09.303905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:09.304613 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:09.310601 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:09.311077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:09.311507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:09.312200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:09.312651 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:09.313089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:09.441668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:09.450306 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ea52f885-57bd-4edf-b1b4-5d706bdb43df tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.151122 Oct 09 01:00:09.450864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 268/534] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:09 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:09.801015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:09.802266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:09.804699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:09.810397 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:09.810883 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:09.811322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:09.812042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:09.812498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:09.812949 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:09.948882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:09.956962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e9c2b34c-ffca-46a2-9ce9-4ceb2467e97d tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.159850 Oct 09 01:00:09.958080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 267/535] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:09 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:10.473670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:10.474273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:10.480587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:10.481068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:10.481505 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:10.482186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:10.482637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:10.483062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:10.614618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:10.622908 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6b5b951e-22f3-454e-acd7-d8c60085cf92 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.153036 Oct 09 01:00:10.623411 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 269/536] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:10 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:10.979903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:10.980511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:10.986821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:10.987302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:10.987773 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:10.988460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:10.989412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:10.989837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:11.117718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:11.125253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2065ef06-f737-485f-ac0b-b86c1e9f69cd tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.149254 Oct 09 01:00:11.125872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 268/537] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:10 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:11.646968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:11.647573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:11.654218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:11.654697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:11.655132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:11.655845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:11.656297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:11.656719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:11.661278 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:11.783786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:11.792499 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d930758b-7883-47f9-86db-d4321d9bf515 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.149930 Oct 09 01:00:11.793065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 270/538] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:11 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:12.148100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:12.148698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:12.155205 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:12.155719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:12.156149 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:12.156825 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:12.157269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:12.157687 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:12.283204 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:12.290764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f5816288-adb5-47d8-9ca4-fa32f8c3b23e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.147156 Oct 09 01:00:12.291317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 269/539] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:12 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:12.816514 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:12.817635 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:12.824086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:12.824402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:12.824832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:12.825538 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:12.825984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:12.826403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:12.961820 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:12.970190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6773ed25-4550-4b0f-8a17-834477e29579 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.157511 Oct 09 01:00:12.970746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 271/540] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:12 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:13.313846 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:13.314449 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:13.320826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:13.321303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:13.321735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:13.322410 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:13.322856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:13.323271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:13.535178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-72283bf0-2a77-460c-bb22-0586f4412cc8 req-46211f78-d3ba-4fa1-bc86-b39af0c9f8ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a6b016ec-cbc1-4ad4-8e80-e79fddc5360d", "name": "network-vif-plugged", "status": "completed", "tag": "48bbfa7d-9313-405d-883a-e594cf9f5314"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:13.545711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-72283bf0-2a77-460c-bb22-0586f4412cc8 req-46211f78-d3ba-4fa1-bc86-b39af0c9f8ce service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:13.546285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-72283bf0-2a77-460c-bb22-0586f4412cc8 req-46211f78-d3ba-4fa1-bc86-b39af0c9f8ce service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:13.546784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-72283bf0-2a77-460c-bb22-0586f4412cc8 req-46211f78-d3ba-4fa1-bc86-b39af0c9f8ce service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:13.560465 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-72283bf0-2a77-460c-bb22-0586f4412cc8 req-46211f78-d3ba-4fa1-bc86-b39af0c9f8ce service nova] Creating event network-vif-plugged:48bbfa7d-9313-405d-883a-e594cf9f5314 for instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d on hv-cinder-maste Oct 09 01:00:13.565422 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-72283bf0-2a77-460c-bb22-0586f4412cc8 req-46211f78-d3ba-4fa1-bc86-b39af0c9f8ce service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034197 Oct 09 01:00:13.566902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 272/541] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:13.672229 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:13.681476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-53818e93-64ac-42e7-a7a2-3319f081b503 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.371768 Oct 09 01:00:13.682135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 270/542] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:13 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:13.994939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:13.995654 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:14.001717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:14.002194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:14.002623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:14.003299 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:14.003782 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:14.004855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:14.133765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:14.141874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ba0c982f-0bff-4d69-9934-3e0288a1c4e7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.151705 Oct 09 01:00:14.142436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 273/543] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:13 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:14.704017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:14.704628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:14.711033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:14.711511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:14.711977 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:14.712666 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:14.713124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:14.713546 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:14.838937 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:14.846491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1fdd211f-404d-442a-b45a-af402a50bead tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1314 microversion: 2.1 time: 0.146339 Oct 09 01:00:14.847053 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 271/544] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:14 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1314 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:15.165959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:15.166586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:15.168973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:15.169613 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:15.175797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:15.176906 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:15.177367 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:15.178040 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:15.178485 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:15.178900 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:15.306521 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:15.314555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cafe6cd3-a7cb-4213-8936-49cd38dc5838 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.152423 Oct 09 01:00:15.315104 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 274/545] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:15 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:15.870528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:15.871128 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:15.872966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:15.878658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:15.879162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:15.879620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:15.880313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:15.880769 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:15.881192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.010362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:16.013787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:16.018985 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-76f9b311-66d6-4ef0-aab0-98dc3c81e9e5 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1463 microversion: 2.1 time: 0.152840 Oct 09 01:00:16.020063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 272/546] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:15 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1463 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:16.065493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:16.065985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:16.072295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:16.072779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:16.073246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.073936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:16.074388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:16.074805 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.100504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:00:16.173846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e0e0833a-cb30-4ecd-9856-484ce7ef5852 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "DELETE /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 204 len: 0 microversion: 2.1 time: 0.111767 Oct 09 01:00:16.175733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 275/547] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:00:16 2022] DELETE /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:00:16.182627 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:16.183239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:16.190830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:16.191456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:16.191940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.192688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:16.193155 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:16.193583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.320351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:16.327913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ec2c5f8c-0783-47ce-aec8-d08edb510c99 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1469 microversion: 2.1 time: 0.149087 Oct 09 01:00:16.328466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 273/548] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:16 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1469 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:16.337923 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:16.338640 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:16.344558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:16.345042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:16.345475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.346158 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:16.346614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:16.347036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:16.472512 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:16.480669 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dd03fb0b-96cd-4639-8571-fb65c740b9c6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.147090 Oct 09 01:00:16.481228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 276/549] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:16 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:17.351253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:17.351909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:17.358222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:17.358705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:17.359138 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:17.359849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:17.360302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:17.360725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:17.489991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:17.497728 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f0400e79-2c2b-4341-8950-a90a696a168e tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1469 microversion: 2.1 time: 0.150472 Oct 09 01:00:17.498286 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 274/550] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:17 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1469 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:17.502951 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:17.503733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:17.510889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:17.511348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:17.511809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:17.512485 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:17.512932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:17.513349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:17.641886 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:17.649847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-09a38c66-d804-43ad-a905-f7caac586fb8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.150732 Oct 09 01:00:17.650403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 277/551] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:17 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:00:18.521520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:18.522158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:18.529418 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:18.530238 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:18.530768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:18.531540 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:18.532095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:18.532593 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:18.554223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-cccd91fe-edfa-4373-bad8-29babcb64e01 req-4cfc7b27-0596-4d09-b6be-b5b36f6e28e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a6b016ec-cbc1-4ad4-8e80-e79fddc5360d", "name": "network-vif-deleted", "tag": "48bbfa7d-9313-405d-883a-e594cf9f5314"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:18.564716 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cccd91fe-edfa-4373-bad8-29babcb64e01 req-4cfc7b27-0596-4d09-b6be-b5b36f6e28e3 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:18.565194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cccd91fe-edfa-4373-bad8-29babcb64e01 req-4cfc7b27-0596-4d09-b6be-b5b36f6e28e3 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:18.565618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cccd91fe-edfa-4373-bad8-29babcb64e01 req-4cfc7b27-0596-4d09-b6be-b5b36f6e28e3 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:18.577895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-cccd91fe-edfa-4373-bad8-29babcb64e01 req-4cfc7b27-0596-4d09-b6be-b5b36f6e28e3 service nova] Creating event network-vif-deleted:48bbfa7d-9313-405d-883a-e594cf9f5314 for instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d on hv-cinder-maste Oct 09 01:00:18.583929 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-cccd91fe-edfa-4373-bad8-29babcb64e01 req-4cfc7b27-0596-4d09-b6be-b5b36f6e28e3 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033461 Oct 09 01:00:18.588106 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 278/552] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:00:18.614039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5becb8eb7b8d4c619f8164de3389a1ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9a6e25abae6b4b499a402bc4585bd06f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:18.622125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bb8a91e0-c798-478d-bba0-26a8125ab3b6 tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 200 len: 1427 microversion: 2.1 time: 0.104867 Oct 09 01:00:18.622710 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 275/553] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:18 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:18.673619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:18.674312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:18.684479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:18.684964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:18.685592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:18.686440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:18.686892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:18.687319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:18.830244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:18.839740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ee397f7d-bcff-4ff8-a097-6db1a4cc1d60 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.170200 Oct 09 01:00:18.840339 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 279/554] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:18 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:19.647360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:19.647938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] [instance: a6b016ec-cbc1-4ad4-8e80-e79fddc5360d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:19.654614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:19.655090 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:19.655527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:19.656244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:19.656701 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:19.657125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:19.670492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] HTTP exception thrown: Instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d could not be found. Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Request method failure captured: Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d HTTP/1.1 Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh0TN1JBbWTJrPK2q_QlPvndlwbPJ7DW5zFvQlfnbzMNC-HdTNvqJKr0LifV0rbNAGxm9LJ5LRFUQscTIrxBOqvdrPGWDJy5Ja_v2ITFpqFfIM3zN-beQrc9Kw7gWMijKa4HuTURadNWy3zYWz2QkKac6GCddSqj9KimgtD5fkLAMvQ Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 9a6e25abae6b4b499a402bc4585bd06f Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesNegativeTest-1245383054 Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:00:19.672105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_9a6e25abae6b4b499a402bc4585bd06f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/9a6e25abae6b4b499a402bc4585bd06f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/9a6e25abae6b4b499a402bc4585bd06f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/9a6e25abae6b4b499a402bc4585bd06f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesNegativeTest-1245383054 Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 9a6e25abae6b4b499a402bc4585bd06f Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesNegativeTest-1245383054 Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesNegativeTest-1245383054-project Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 5becb8eb7b8d4c619f8164de3389a1ce Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesNegativeTest-1245383054-project Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d could not be found. Oct 09 01:00:19.673467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'a6b016ec-cbc1-4ad4-8e80-e79fddc5360d'} Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d could not be found. Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:00:19.674193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d could not be found. Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d could not be found. Oct 09 01:00:19.675789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:00:19.677625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] Returning 404 to user: Instance a6b016ec-cbc1-4ad4-8e80-e79fddc5360d could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:00:19.677625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-81749d7f-0e4d-4c1f-ad5f-0a338d0b709c tempest-VolumesNegativeTest-1245383054 tempest-VolumesNegativeTest-1245383054-project] 10.222.0.33 "GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d" status: 404 len: 111 microversion: 2.1 time: 0.030015 Oct 09 01:00:19.677625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 276/555] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:19 2022] GET /compute/v2.1/servers/a6b016ec-cbc1-4ad4-8e80-e79fddc5360d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:00:19.863260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:19.863897 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:19.871055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:19.871541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:19.872011 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:19.872748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:19.873206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:19.873627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:20.006933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:20.015359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-95346d7a-113a-4451-8589-e67641428e6f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.155914 Oct 09 01:00:20.015947 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 280/556] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:19 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:21.040141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:21.040849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:21.047987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:21.048599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:21.049127 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:21.049884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:21.050452 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:21.051500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:21.184763 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:21.193237 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-02eba74e-8d34-4e85-bd22-016ecfadab87 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.156968 Oct 09 01:00:21.193866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 277/557] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:21 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:00:22.138704 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:22.140918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:22.147911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.148391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.149201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.149891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.150346 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.150762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.184660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5b940f1e-8e84-4806-815c-6f4b23afb7d6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049968 Oct 09 01:00:22.185352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 281/558] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:00:22 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:00:22.217279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:22.217887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:22.225044 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.225538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.225980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.226879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.227339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.227808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.273449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:22.273951 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:22.290193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.290688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.291120 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.291851 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.292298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.292737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.331184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:00:22.385065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:22.393786 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-236191f8-53e6-4312-8296-878cf00bf692 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.180258 Oct 09 01:00:22.394885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 278/559] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:22 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:22.407372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f89c5bc0-5c36-4bd4-bfcc-ddc66b905883 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 204 len: 0 microversion: 2.1 time: 0.149644 Oct 09 01:00:22.408085 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 282/560] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:00:22 2022] DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 09 01:00:22.416517 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:22.417038 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:22.423162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.423668 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.424102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.424783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:22.425228 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:22.425688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:22.554103 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:22.561958 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8e4b5fbc-78e5-4405-8b67-ec46b7c25fef tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1673 microversion: 2.1 time: 0.149474 Oct 09 01:00:22.562632 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 279/561] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:22 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1673 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:23.418815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:23.419491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:23.426329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:23.426805 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:23.427232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:23.427934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:23.428383 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:23.428799 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:23.564845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:23.573506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dc502d4d-2cad-402d-b98d-6125d2e8cd22 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.158790 Oct 09 01:00:23.574069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 283/562] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:23 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:23.586353 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:23.586986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:23.593624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:23.594106 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:23.594538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:23.595306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:23.595722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:23.596168 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:23.743612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:23.751205 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-968e898b-d54e-45f6-a659-b3f708a0da67 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1673 microversion: 2.1 time: 0.169677 Oct 09 01:00:23.751814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 280/563] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:23 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1673 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:24.598496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:24.599131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:24.605557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:24.606036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:24.606471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:24.607149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:24.607618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:24.608043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:24.736400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:24.739328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:24.747128 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b24131b7-ad79-4297-8762-6cb425a61b03 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.153075 Oct 09 01:00:24.747725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 284/564] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:24 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:24.774226 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:24.774871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:24.781302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:24.781781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:24.782248 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:24.782965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:24.783416 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:24.783862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:24.916415 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:24.924050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fd7b4254-52b0-4d16-87e1-6a924b2bc37d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1673 microversion: 2.1 time: 0.153599 Oct 09 01:00:24.924612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 281/565] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:24 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1673 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:25.565545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-a665191a-a8f4-4cf7-b660-e1263b21ed96 req-31a74370-869b-40c7-bef1-fec15154375b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3247d2ef-8e30-4cee-b097-1957a0a3de81", "name": "network-vif-deleted", "tag": "035fc085-892c-4866-9ff4-6bd3d7193e9b"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:25.576992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a665191a-a8f4-4cf7-b660-e1263b21ed96 req-31a74370-869b-40c7-bef1-fec15154375b service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:25.577829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a665191a-a8f4-4cf7-b660-e1263b21ed96 req-31a74370-869b-40c7-bef1-fec15154375b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:25.578460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a665191a-a8f4-4cf7-b660-e1263b21ed96 req-31a74370-869b-40c7-bef1-fec15154375b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:25.592175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-a665191a-a8f4-4cf7-b660-e1263b21ed96 req-31a74370-869b-40c7-bef1-fec15154375b service nova] Creating event network-vif-deleted:035fc085-892c-4866-9ff4-6bd3d7193e9b for instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 on hv-cinder-maste Oct 09 01:00:25.598066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-a665191a-a8f4-4cf7-b660-e1263b21ed96 req-31a74370-869b-40c7-bef1-fec15154375b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036036 Oct 09 01:00:25.602770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 285/566] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:25.771785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:25.772777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:25.780711 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:25.782085 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:25.782879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:25.784107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:25.785456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:25.786125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:25.950174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:25.950867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:25.958089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:25.959200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:25.959823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:25.960700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:25.961421 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:25.962078 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:25.976410 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:25.985564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3cb85665-f2d5-4357-9905-50dbc74d6493 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.218964 Oct 09 01:00:25.986141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 282/567] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:25 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:25.999580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:26.000329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:26.000948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:26.001995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lazy-loading 'fault' on Instance uuid 3247d2ef-8e30-4cee-b097-1957a0a3de81 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:00:26.056088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:26.064377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8ef1d277-11aa-4a99-8dae-967be5f16d53 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1327 microversion: 2.1 time: 0.120529 Oct 09 01:00:26.064943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 286/568] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:25 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1327 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:00:27.008604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:27.009249 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:27.015036 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:27.015511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:27.015981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:27.016658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:27.017104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:27.017521 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:27.088007 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:27.088639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:27.094961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:27.096138 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:27.096612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:27.097997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:27.098463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:27.098884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:27.143419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:27.143933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:27.144379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:27.144838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lazy-loading 'fault' on Instance uuid 3247d2ef-8e30-4cee-b097-1957a0a3de81 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:00:27.154391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:27.181351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-709fafed-1520-4ff2-8141-24b68e2da68d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1734 microversion: 2.1 time: 0.173106 Oct 09 01:00:27.181981 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 283/569] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:27 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1734 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:27.199063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:27.209126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e965e98a-622c-4424-a6b2-43e5e1d88dfc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 200 len: 1327 microversion: 2.1 time: 0.124981 Oct 09 01:00:27.209677 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 287/570] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:27 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 1327 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:00:28.209143 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:28.209733 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:28.216267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.216762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.217194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.217871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.218348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.218764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.231574 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:28.232353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:28.238672 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.239148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.239577 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.240274 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.240719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.241138 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.256446 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] HTTP exception thrown: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Request method failure captured: Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 HTTP/1.1 Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwalz5AigjQALstJl8FsJmuzD3g_ClB_VYUuUUBcr-err98-DsiSVUJM2XJMdcTEVZoUZCtrt9Dg5SkbdtyQ2v5kfPi91WnRH5l3IHEmhLcsHiTfQgJqyt2C8bqkh8eQaUJRBi2s49vd5c_FIxo3rJE63U7Ut34lA5pMGq0cWwewEI Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:00:28.258070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_40e631c5e320489cac9cea85118c6263"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/40e631c5e320489cac9cea85118c6263"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 1512c91201f14b75a9be35b89c2097ef Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:00:28.259726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '3247d2ef-8e30-4cee-b097-1957a0a3de81'} Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:00:28.260695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:00:28.261893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.262965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Returning 404 to user: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:00:28.262965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f2115628-dcfb-4e94-a169-6e8ef7575ec9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 404 len: 111 microversion: 2.1 time: 0.031624 Oct 09 01:00:28.262965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 288/571] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:28 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 01:00:28.268151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bbb541e0-6e18-431c-bd53-825e3d5c9b9e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '.version_select at 0x7f7268b2fee0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:28.287985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bbb541e0-6e18-431c-bd53-825e3d5c9b9e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1091593546" status: 202 len: 0 microversion: 2.1 time: 0.023365 Oct 09 01:00:28.289384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 289/572] 10.222.0.33 () {58 vars in 1221 bytes} [Sun Oct 9 01:00:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1091593546 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:00:28.394748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:28.403132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f9852912-b330-4787-a2e2-522b57d0642c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1714 microversion: 2.1 time: 0.198504 Oct 09 01:00:28.403713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 284/573] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:28 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1714 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:28.520491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:28.520971 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:28.527089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.527568 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.528033 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.528709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.529152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.529590 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.652092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] HTTP exception thrown: Cannot 'detach_volume' instance 50224067-d3e0-4567-bc63-31ace3a211c6 while it is in vm_state rescued Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Request method failure captured: Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments/af3f6b36-1fc1-4dde-9076-c1ec90e38994 HTTP/1.1 Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwpCa7qTsGCbgYzI3hTp0XV4aX7UY9kYcqRmcLmnNuVu0pwn81JmE0Q7Hu-M48YX0Ivzl80hJOivFomkM56cwHMuXcUttfPVYuxDmo1TXJ7o2oEOERlIn_KBndSYXeiVrB5ti0ZjjXNGyRZ2bEWEYWMfg6VZDBo93QOHqW60b7MagU Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:00:28.653565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 438446d1e6b3402bb18b547c089391f8 Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:00:28.655117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Cannot 'detach_volume' instance 50224067-d3e0-4567-bc63-31ace3a211c6 while it is in vm_state rescued Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'server_id': '50224067-d3e0-4567-bc63-31ace3a211c6', 'id': 'af3f6b36-1fc1-4dde-9076-c1ec90e38994'} Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:00:28.655887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 50224067-d3e0-4567-bc63-31ace3a211c6 while it is in vm_state rescued Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 50224067-d3e0-4567-bc63-31ace3a211c6 while it is in vm_state rescued Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:28.657074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Returning 409 to user: Cannot 'detach_volume' instance 50224067-d3e0-4567-bc63-31ace3a211c6 while it is in vm_state rescued {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:00:28.658180 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-aef5a5a8-11ac-4732-b8bc-d588cdb3c15b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments/af3f6b36-1fc1-4dde-9076-c1ec90e38994" status: 409 len: 152 microversion: 2.1 time: 0.139277 Oct 09 01:00:28.658180 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 290/574] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:00:28 2022] DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments/af3f6b36-1fc1-4dde-9076-c1ec90e38994 => generated 152 bytes in 140 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 09 01:00:28.664233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:28.664767 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:28.670601 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.671086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.671531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.672313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.672765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.673198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.744034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d50196f2-8fd9-48c9-8c82-6b28a10d3b08 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "POST /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/action" status: 202 len: 0 microversion: 2.1 time: 0.083396 Oct 09 01:00:28.747586 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 285/575] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 01:00:28 2022] POST /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/action => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:00:28.752838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:28.753988 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:28.756768 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=3,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:28.762337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.763026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.763463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.764203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:28.764659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:28.765098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:28.980857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-40b71d4d-abcc-44a9-8121-44fbe1d353eb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: .version_select at 0x7f7268a4c8b0>, body: {"keypair": {"name": "tempest-keypair-1646329788", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG7uSdkqI1JwnY46sCLgOC76djHzl87olk3A2j0YU0W452w/+GiueKSWbkFqAqhaRItakPXer7bPCHYw8ueR16Ys3mqr3fWsSUNkbg5zcsmwlXQ7j45YqAwWKKm+wcp6VQ=="}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:28.989022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:28.996866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-40b71d4d-abcc-44a9-8121-44fbe1d353eb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for project 40e631c5e320489cac9cea85118c6263. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:00:28.997653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fd688482-f827-4514-ad25-950b5991164e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1722 microversion: 2.1 time: 0.248523 Oct 09 01:00:28.998319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 291/576] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:28 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1722 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:29.011820 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-40b71d4d-abcc-44a9-8121-44fbe1d353eb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for user 1512c91201f14b75a9be35b89c2097ef and project 40e631c5e320489cac9cea85118c6263. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:00:29.023538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-40b71d4d-abcc-44a9-8121-44fbe1d353eb tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047005 Oct 09 01:00:29.024176 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 286/577] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 01:00:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:30.021762 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:30.022481 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:30.029016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:30.029572 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:30.030077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:30.030838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:30.031364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:30.031908 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:30.167243 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:30.175713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e410710f-1ae9-4ff4-bcc9-7e7e92b464e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1722 microversion: 2.1 time: 0.157985 Oct 09 01:00:30.176279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 292/578] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:30 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1722 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:30.309004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1363258475", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "93a9ba3d-37f0-48ce-9786-b421e5404102"}], "security_groups": [{"name": "tempest-securitygroup--477921715"}], "key_name": "tempest-keypair-1646329788", "user_data": "}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:30.411709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] validate_networks() for [('93a9ba3d-37f0-48ce-9786-b421e5404102', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:00:30.774705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:00:30.775172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:00:30.775636 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:00:30.776063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:00:30.776441 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:00:30.785494 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:30.785852 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:30.786284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:30.787003 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:30.787456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:30.787919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:30.804482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for project 40e631c5e320489cac9cea85118c6263. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:00:30.808443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Getting quotas for user 1512c91201f14b75a9be35b89c2097ef and project 40e631c5e320489cac9cea85118c6263. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:00:30.820105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:00:30.822452 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:00:30.869465 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6dbfff62-7342-45fa-a865-976d909bcbb9 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.564787 Oct 09 01:00:30.870100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 287/579] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:00:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 566 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:00:30.882547 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-aedeae24-cebb-4aa6-83a7-4ea4b0b04752 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:30.883155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-aedeae24-cebb-4aa6-83a7-4ea4b0b04752 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:30.935216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-aedeae24-cebb-4aa6-83a7-4ea4b0b04752 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:30.942788 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-aedeae24-cebb-4aa6-83a7-4ea4b0b04752 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1238 microversion: 2.1 time: 0.064637 Oct 09 01:00:30.943384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 293/580] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:30 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1238 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:31.199556 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:31.200258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:31.207801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:31.208296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:31.208740 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:31.209443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:31.209901 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:31.210329 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:31.345534 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:31.354232 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-21f4f225-0a43-4d20-824b-3ed7edf05d53 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1722 microversion: 2.1 time: 0.158822 Oct 09 01:00:31.354787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 288/581] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:31 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:31.964765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:31.965364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:31.973122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:31.974002 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:31.974434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:31.975113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:31.975561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:31.976007 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:32.036945 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:32.044628 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9a1ed98f-f6b8-4900-ada2-f1bf7304c8bc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1234 microversion: 2.1 time: 0.083679 Oct 09 01:00:32.045188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 294/582] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:31 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1234 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:32.377170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:32.377776 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:32.384610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:32.385091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:32.385528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:32.386212 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:32.386665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:32.387091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:32.513686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:32.522420 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1fe36a38-f77e-42ac-8b1f-76cf67a31aa7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1728 microversion: 2.1 time: 0.149110 Oct 09 01:00:32.522978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 289/583] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:32 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1728 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:32.545708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:32.546518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:32.553518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:32.553998 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:32.554428 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:32.555108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:32.555552 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:32.555997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:32.636935 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f7eb3478-b4cb-4eb2-a474-3b44940ae9be tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.095793 Oct 09 01:00:32.637528 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 295/584] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 01:00:32 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-interface => generated 284 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:33.066594 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:33.067404 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:33.086051 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:33.086572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:33.087082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:33.087817 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:33.088278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:33.088712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:33.158055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:33.162940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:33.168095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e81760b3-d2b0-44f1-8701-246bfc86dadc tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1296 microversion: 2.1 time: 0.105603 Oct 09 01:00:33.168712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 290/585] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:33 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1296 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:33.599148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-7cc3a354-d03a-4b70-853e-67f1b27da7e0 req-1dbe8288-7c12-4d0f-9a31-6deabdb458d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50224067-d3e0-4567-bc63-31ace3a211c6", "tag": "af11de25-9b71-43af-8ce0-051a32ec945c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:33.617508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7cc3a354-d03a-4b70-853e-67f1b27da7e0 req-1dbe8288-7c12-4d0f-9a31-6deabdb458d2 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:33.622884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7cc3a354-d03a-4b70-853e-67f1b27da7e0 req-1dbe8288-7c12-4d0f-9a31-6deabdb458d2 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:33.624269 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7cc3a354-d03a-4b70-853e-67f1b27da7e0 req-1dbe8288-7c12-4d0f-9a31-6deabdb458d2 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:33.640301 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-7cc3a354-d03a-4b70-853e-67f1b27da7e0 req-1dbe8288-7c12-4d0f-9a31-6deabdb458d2 service nova] Creating event network-changed:af11de25-9b71-43af-8ce0-051a32ec945c for instance 50224067-d3e0-4567-bc63-31ace3a211c6 on hv-cinder-maste Oct 09 01:00:33.645663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-7cc3a354-d03a-4b70-853e-67f1b27da7e0 req-1dbe8288-7c12-4d0f-9a31-6deabdb458d2 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050868 Oct 09 01:00:33.646661 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 296/586] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:34.192359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:34.193570 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:34.204669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:34.205156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:34.205845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:34.207718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:34.208898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:34.208898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:34.361016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:34.369533 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e3d87556-919c-45cb-a0d4-c9c824e420f6 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.181106 Oct 09 01:00:34.370943 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 291/587] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:34 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:34.608776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-a6981e80-9ba5-4d3b-a44f-348c68ccafdb req-f934076e-b20a-49d1-b98c-b989a64ebc5c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bf1a22b3-a821-421e-8d9e-93190101a78d", "tag": "0b9a0bca-1f0d-4dc1-a233-b41357e21184"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:34.618766 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a6981e80-9ba5-4d3b-a44f-348c68ccafdb req-f934076e-b20a-49d1-b98c-b989a64ebc5c service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:34.619232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a6981e80-9ba5-4d3b-a44f-348c68ccafdb req-f934076e-b20a-49d1-b98c-b989a64ebc5c service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:34.619684 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a6981e80-9ba5-4d3b-a44f-348c68ccafdb req-f934076e-b20a-49d1-b98c-b989a64ebc5c service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:34.633526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-a6981e80-9ba5-4d3b-a44f-348c68ccafdb req-f934076e-b20a-49d1-b98c-b989a64ebc5c service nova] Creating event network-changed:0b9a0bca-1f0d-4dc1-a233-b41357e21184 for instance bf1a22b3-a821-421e-8d9e-93190101a78d on hv-cinder-maste Oct 09 01:00:34.638139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-a6981e80-9ba5-4d3b-a44f-348c68ccafdb req-f934076e-b20a-49d1-b98c-b989a64ebc5c service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033576 Oct 09 01:00:34.638734 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 297/588] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:35.393547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:35.394668 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:35.401371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:35.402103 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:35.402535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:35.403215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:35.403693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:35.404111 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:35.532315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:35.540789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a2774a6f-9c86-49e6-8c51-43a92a414aca tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.151061 Oct 09 01:00:35.541358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 292/589] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:35 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:35.661487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-ce8d9147-b88a-4c5d-b79c-f7b08a3b1a7b req-4738c820-c22c-40a7-a6ce-ac730774a751 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "50224067-d3e0-4567-bc63-31ace3a211c6", "tag": "af11de25-9b71-43af-8ce0-051a32ec945c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:35.670793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ce8d9147-b88a-4c5d-b79c-f7b08a3b1a7b req-4738c820-c22c-40a7-a6ce-ac730774a751 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:35.671267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ce8d9147-b88a-4c5d-b79c-f7b08a3b1a7b req-4738c820-c22c-40a7-a6ce-ac730774a751 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:35.671721 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ce8d9147-b88a-4c5d-b79c-f7b08a3b1a7b req-4738c820-c22c-40a7-a6ce-ac730774a751 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:35.682529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-ce8d9147-b88a-4c5d-b79c-f7b08a3b1a7b req-4738c820-c22c-40a7-a6ce-ac730774a751 service nova] Creating event network-changed:af11de25-9b71-43af-8ce0-051a32ec945c for instance 50224067-d3e0-4567-bc63-31ace3a211c6 on hv-cinder-maste Oct 09 01:00:35.687304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-ce8d9147-b88a-4c5d-b79c-f7b08a3b1a7b req-4738c820-c22c-40a7-a6ce-ac730774a751 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030421 Oct 09 01:00:35.687891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 298/590] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:00:35.739025 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:35.739505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:35.745388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:35.745863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:35.746292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:35.746961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:35.747408 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:35.747853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:35.770742 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bab52984-6b14-4d74-8df0-96d04b11f991 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/ips" status: 200 len: 151 microversion: 2.1 time: 0.036014 Oct 09 01:00:35.771280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 293/591] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 01:00:35 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/ips => generated 151 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:36.564271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:36.564966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:36.571534 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:36.572050 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:36.572498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:36.573167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:36.573617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:36.574053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:36.865977 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:36.873708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f8391df1-65e5-4378-a2b3-576a36edd4f2 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.313645 Oct 09 01:00:36.874269 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 299/592] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:36 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:37.100279 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c2da79fb-2874-4019-bc6e-79354dbbcaf2"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:37.102765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:37.109432 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:37.109922 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:37.110360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:37.111059 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:37.111519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:37.111966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:37.403098 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1689e697-a75f-4d45-a01d-686f435555bf tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.306495 Oct 09 01:00:37.407165 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 294/593] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:00:37 2022] POST /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments => generated 194 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:37.897587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:37.898154 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:37.904703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:37.905185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:37.905619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:37.906303 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:37.906756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:37.907188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:38.033207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:38.041778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9a2fe33c-5c6f-424f-895c-f082ec60aced tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.148686 Oct 09 01:00:38.042343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 300/594] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:37 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:39.065493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:39.066101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:39.072192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:39.072678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:39.073148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:39.073833 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:39.074278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:39.074699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:39.200988 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:39.209011 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-02134478-7978-4bfa-8117-453182395a8d tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.148357 Oct 09 01:00:39.209560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 295/595] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:39 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:39.614384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-ee8a0317-d377-42d0-aadb-4ea107df847d req-768c3b3e-27c7-4b87-9901-8185c1ad65bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bf1a22b3-a821-421e-8d9e-93190101a78d", "name": "network-vif-plugged", "status": "completed", "tag": "0b9a0bca-1f0d-4dc1-a233-b41357e21184"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:39.624275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ee8a0317-d377-42d0-aadb-4ea107df847d req-768c3b3e-27c7-4b87-9901-8185c1ad65bd service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:39.624756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ee8a0317-d377-42d0-aadb-4ea107df847d req-768c3b3e-27c7-4b87-9901-8185c1ad65bd service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:39.625190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-ee8a0317-d377-42d0-aadb-4ea107df847d req-768c3b3e-27c7-4b87-9901-8185c1ad65bd service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:39.636722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-ee8a0317-d377-42d0-aadb-4ea107df847d req-768c3b3e-27c7-4b87-9901-8185c1ad65bd service nova] Creating event network-vif-plugged:0b9a0bca-1f0d-4dc1-a233-b41357e21184 for instance bf1a22b3-a821-421e-8d9e-93190101a78d on hv-cinder-maste Oct 09 01:00:39.641409 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-ee8a0317-d377-42d0-aadb-4ea107df847d req-768c3b3e-27c7-4b87-9901-8185c1ad65bd service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030727 Oct 09 01:00:39.642048 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 301/596] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:40.231753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:40.232387 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:40.234762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=4 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:40.241132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:40.241741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:40.242177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:40.242856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:40.243301 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:40.243895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:40.375406 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:40.382961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a5f2eae4-5211-4fcc-b5ee-2b6e1911855c tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.154991 Oct 09 01:00:40.383509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 296/597] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:40 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:41.402685 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:41.406905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:41.407617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:41.413935 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:41.414421 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:41.414859 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:41.415577 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:41.416078 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:41.416519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:41.546640 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:41.554744 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0eeaf20b-ac7d-4ee6-ba58-bf21b0902a3a tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1363 microversion: 2.1 time: 0.153701 Oct 09 01:00:41.555296 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 302/598] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:41 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1363 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:42.577752 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:42.578396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:42.580695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:42.581990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=10 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:42.587970 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:42.588459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:42.588900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:42.589589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:42.590043 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:42.590475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:42.716471 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:42.724166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4b2a0c50-438d-49c5-9175-570ed64e05f0 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1549 microversion: 2.1 time: 0.150265 Oct 09 01:00:42.724949 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 297/599] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:42 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1549 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:42.746062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:42.746564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:42.752945 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:42.753941 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:42.754377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:42.755070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:42.755523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:42.756001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:42.836096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-823a6819-d4c7-44fa-8e67-7eb102207062 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.093732 Oct 09 01:00:42.836696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 303/600] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 01:00:42 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:43.750802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-c3da810d-1477-4464-8b34-3fc5d8a486b2 req-7ec78888-bc30-4535-874d-c5d0887124db service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bf1a22b3-a821-421e-8d9e-93190101a78d", "tag": "0b9a0bca-1f0d-4dc1-a233-b41357e21184"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:43.760804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-c3da810d-1477-4464-8b34-3fc5d8a486b2 req-7ec78888-bc30-4535-874d-c5d0887124db service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:43.761270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-c3da810d-1477-4464-8b34-3fc5d8a486b2 req-7ec78888-bc30-4535-874d-c5d0887124db service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:43.761691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-c3da810d-1477-4464-8b34-3fc5d8a486b2 req-7ec78888-bc30-4535-874d-c5d0887124db service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:43.773386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-c3da810d-1477-4464-8b34-3fc5d8a486b2 req-7ec78888-bc30-4535-874d-c5d0887124db service nova] Creating event network-changed:0b9a0bca-1f0d-4dc1-a233-b41357e21184 for instance bf1a22b3-a821-421e-8d9e-93190101a78d on hv-cinder-maste Oct 09 01:00:43.778751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-c3da810d-1477-4464-8b34-3fc5d8a486b2 req-7ec78888-bc30-4535-874d-c5d0887124db service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031501 Oct 09 01:00:43.779344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 298/601] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:00:44.946042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c2da79fb-2874-4019-bc6e-79354dbbcaf2"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:44.948392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:44.954315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:44.954800 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:44.955241 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:44.955958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:44.956474 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:44.956900 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:45.034738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Invalid volume: volume c2da79fb-2874-4019-bc6e-79354dbbcaf2 is already attached to instances: 188a0b8d-23e2-4769-94bf-85277c3b0b66 Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: POST /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments HTTP/1.1 Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 74 Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:00:45.036739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {"volumeAttachment": {"volumeId": "c2da79fb-2874-4019-bc6e-79354dbbcaf2"}} Oct 09 01:00:45.038283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Invalid volume: volume c2da79fb-2874-4019-bc6e-79354dbbcaf2 is already attached to instances: 188a0b8d-23e2-4769-94bf-85277c3b0b66 Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'server_id': '188a0b8d-23e2-4769-94bf-85277c3b0b66', 'body': {'volumeAttachment': {'volumeId': 'c2da79fb-2874-4019-bc6e-79354dbbcaf2'}}} Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume c2da79fb-2874-4019-bc6e-79354dbbcaf2 is already attached to instances: 188a0b8d-23e2-4769-94bf-85277c3b0b66 Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:45.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume c2da79fb-2874-4019-bc6e-79354dbbcaf2 is already attached to instances: 188a0b8d-23e2-4769-94bf-85277c3b0b66 Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:45.040508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume c2da79fb-2874-4019-bc6e-79354dbbcaf2 is already attached to instances: 188a0b8d-23e2-4769-94bf-85277c3b0b66 Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 400 to user: Invalid volume: volume c2da79fb-2874-4019-bc6e-79354dbbcaf2 is already attached to instances: 188a0b8d-23e2-4769-94bf-85277c3b0b66 {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-35c692a7-006d-4434-8c72-c8a254bfaf1d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.096744 Oct 09 01:00:45.041982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 304/602] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:00:44 2022] POST /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments => generated 174 bytes in 98 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:00:45.104577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:45.105150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:45.111192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:45.111712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:45.112149 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:45.113469 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:45.113925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:45.114357 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:45.194759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'services' on Instance uuid 188a0b8d-23e2-4769-94bf-85277c3b0b66 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:00:45.263163 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-be8f2417-83fa-426b-a914-30558887e90b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments/c2da79fb-2874-4019-bc6e-79354dbbcaf2" status: 202 len: 0 microversion: 2.1 time: 0.163102 Oct 09 01:00:45.264109 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 299/603] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:00:45 2022] DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments/c2da79fb-2874-4019-bc6e-79354dbbcaf2 => generated 0 bytes in 165 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:00:45.273070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:45.275332 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:45.281791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:45.282301 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:45.282716 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:45.283419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:45.283909 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:45.284892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:45.316250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ae10b984-753d-4520-989a-ee651ad0f0fa tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047685 Oct 09 01:00:45.316809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 305/604] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:00:45 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:00:46.329318 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:46.331612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:46.338251 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:46.338661 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:46.339093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:46.339933 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:46.340381 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:46.340800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:46.369785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ce9facca-e9bd-4228-9fc0-649656b79c9f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.044781 Oct 09 01:00:46.370320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 300/605] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:00:46 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:00:46.432198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:46.432705 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:46.438926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:46.439397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:46.439853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:46.440519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:46.440956 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:46.441369 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:46.465093 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:00:46.488625 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:46.538966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f8c4acf4-6696-46e0-891f-f5b0d9e80a12 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 204 len: 0 microversion: 2.1 time: 0.110191 Oct 09 01:00:46.539603 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 306/606] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:00:46 2022] DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:00:46.547841 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:46.548443 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:46.554898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:46.555373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:46.555839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:46.556516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:46.556982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:46.557378 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:46.687853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:46.695466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-96986a31-9fc3-4c1d-bffc-a0490ce5d685 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1683 microversion: 2.1 time: 0.151366 Oct 09 01:00:46.696048 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 301/607] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:46 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1683 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:47.718954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:47.719671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:47.725992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:47.726473 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:47.726908 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:47.727607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:47.728059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:47.728485 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:47.856657 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:47.864725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e7c95b8d-571d-436b-822c-99ab5866c3d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1683 microversion: 2.1 time: 0.150017 Oct 09 01:00:47.865287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 307/608] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:47 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1683 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:48.890426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:48.890964 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:48.898184 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:48.898666 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:48.899103 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:48.899887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:48.900343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:48.900768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:49.029830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:49.037594 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-eae7b75c-549d-4cfd-bb85-54813594293f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1683 microversion: 2.1 time: 0.151137 Oct 09 01:00:49.038154 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 302/609] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:48 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1683 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:49.405936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-cf4b5c21-8b46-4d6a-ac8a-95c847a728b0 req-56fe210a-cc5c-4de8-9f79-6ebad647e396 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "188a0b8d-23e2-4769-94bf-85277c3b0b66", "name": "network-vif-deleted", "tag": "29a59039-ac16-488f-ad9d-1a896696e3f6"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:49.416533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cf4b5c21-8b46-4d6a-ac8a-95c847a728b0 req-56fe210a-cc5c-4de8-9f79-6ebad647e396 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:49.417090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cf4b5c21-8b46-4d6a-ac8a-95c847a728b0 req-56fe210a-cc5c-4de8-9f79-6ebad647e396 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:49.417523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cf4b5c21-8b46-4d6a-ac8a-95c847a728b0 req-56fe210a-cc5c-4de8-9f79-6ebad647e396 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:49.428845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-cf4b5c21-8b46-4d6a-ac8a-95c847a728b0 req-56fe210a-cc5c-4de8-9f79-6ebad647e396 service nova] Creating event network-vif-deleted:29a59039-ac16-488f-ad9d-1a896696e3f6 for instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 on hv-cinder-maste Oct 09 01:00:49.433955 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-cf4b5c21-8b46-4d6a-ac8a-95c847a728b0 req-56fe210a-cc5c-4de8-9f79-6ebad647e396 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032480 Oct 09 01:00:49.435850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 308/610] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:50.060610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:50.061582 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:50.068910 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:50.069764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:50.069899 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:50.070779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:50.071310 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:50.071851 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:50.108213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:50.108803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:50.109248 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:50.109699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'fault' on Instance uuid 188a0b8d-23e2-4769-94bf-85277c3b0b66 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:00:50.154170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:50.162776 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-83ab5c53-91e2-4b06-ae46-c1fda61f6456 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 200 len: 1332 microversion: 2.1 time: 0.105983 Oct 09 01:00:50.163351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 303/611] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:50 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 1332 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:51.187189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:51.187935 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:51.194627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:51.195107 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:51.195835 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:51.196283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:51.196733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:51.197151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:51.210669 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 HTTP/1.1 Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:00:51.212330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:00:51.213911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '188a0b8d-23e2-4769-94bf-85277c3b0b66'} Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:00:51.214864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:00:51.216380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:00:51.217747 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:00:51.217747 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d0eee266-53e9-491a-8848-df1d339f772a tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 404 len: 111 microversion: 2.1 time: 0.031174 Oct 09 01:00:51.217747 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 309/612] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:51 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:00:51.222675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a39bc780-fd52-4592-a466-c751829d858c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '.version_select at 0x7f7268a0ac10>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:51.241060 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a39bc780-fd52-4592-a466-c751829d858c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2048709389" status: 202 len: 0 microversion: 2.1 time: 0.022222 Oct 09 01:00:51.241604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 304/613] 10.222.0.33 () {58 vars in 1221 bytes} [Sun Oct 9 01:00:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2048709389 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:00:52.029446 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-89f01940-1201-426e-be8c-6890e09f24d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: .version_select at 0x7f7268a6df70>, body: {"keypair": {"name": "tempest-keypair-33835709", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPFYenvpPTbKzNr0NpJVBVETD5jacvrzk7h+u3mENrGsI/9PW0WINMY4WbT3kbp9rGEgscy3GjpT/CP2Hh0QpgcFzgd8nbQOPkTQ4FA4criUPQZ6G+/p/5Q9wCIfk/YNMQ=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:52.044785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-89f01940-1201-426e-be8c-6890e09f24d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:00:52.048451 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-89f01940-1201-426e-be8c-6890e09f24d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:00:52.068726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-89f01940-1201-426e-be8c-6890e09f24d9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.043434 Oct 09 01:00:52.069317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 310/614] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 01:00:52 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:00:53.301468 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1257730039", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ef6b019b-3ff0-42c3-b2bd-354f2c856ccc"}], "security_groups": [{"name": "tempest-securitygroup--769657378"}], "key_name": "tempest-keypair-33835709", "user_data": "}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:53.554190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] validate_networks() for [('ef6b019b-3ff0-42c3-b2bd-354f2c856ccc', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:00:53.916383 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:00:53.916834 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:00:53.917303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:00:53.917737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:00:53.918170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:00:53.927567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:53.928069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:53.928501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:53.929216 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:53.929664 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:53.930083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:53.946545 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:00:53.950491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Getting quotas for user b027fb8adadc488e975c4541e7ee6626 and project a7d8e82f58814da6b8c08bfa04955d4c. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:00:53.961264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:00:53.963518 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:00:54.011177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8e299732-f688-4367-b0b3-fd8fbbcd62a5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.715206 Oct 09 01:00:54.011848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 305/615] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:00:53 2022] POST /compute/v2.1/servers => generated 401 bytes in 717 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:00:54.023337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-49fb5c70-b2bc-49b2-9ff0-618e9fb59fec tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:54.023964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-49fb5c70-b2bc-49b2-9ff0-618e9fb59fec tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:54.448316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-49fb5c70-b2bc-49b2-9ff0-618e9fb59fec tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:54.456985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-49fb5c70-b2bc-49b2-9ff0-618e9fb59fec tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1240 microversion: 2.1 time: 0.437883 Oct 09 01:00:54.458057 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 311/616] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:54 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1240 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:55.480358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:55.480972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:55.488055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:55.488531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:55.488967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:55.489653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:55.490104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:55.490525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:55.555246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:55.562987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2c2cbb70-eae0-46f7-8280-bc694a826506 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1300 microversion: 2.1 time: 0.086549 Oct 09 01:00:55.563544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 306/617] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:55 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1300 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:56.586198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:56.586798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:56.593352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:56.593759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:56.594187 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:56.594863 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:56.595308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:56.595756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:56.834972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:56.842843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ab087490-1aa6-4a12-856b-6cb520890397 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.260537 Oct 09 01:00:56.843466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 312/618] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:56 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:57.225900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-db4947bb-9be1-4b00-9b80-f47c680bb385 req-ccf9436e-9c3b-4311-aed9-924c8e88023e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e0d9a7c4-b33c-42c1-9042-dfd357bc3932", "tag": "bf1ecde5-a035-439a-96b4-f7a4880b70e4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:00:57.236839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-db4947bb-9be1-4b00-9b80-f47c680bb385 req-ccf9436e-9c3b-4311-aed9-924c8e88023e service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:57.237318 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-db4947bb-9be1-4b00-9b80-f47c680bb385 req-ccf9436e-9c3b-4311-aed9-924c8e88023e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:57.237799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-db4947bb-9be1-4b00-9b80-f47c680bb385 req-ccf9436e-9c3b-4311-aed9-924c8e88023e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:57.248664 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-db4947bb-9be1-4b00-9b80-f47c680bb385 req-ccf9436e-9c3b-4311-aed9-924c8e88023e service nova] Creating event network-changed:bf1ecde5-a035-439a-96b4-f7a4880b70e4 for instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 on hv-cinder-maste Oct 09 01:00:57.253525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-db4947bb-9be1-4b00-9b80-f47c680bb385 req-ccf9436e-9c3b-4311-aed9-924c8e88023e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031966 Oct 09 01:00:57.254115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 307/619] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:00:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:00:57.866252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:57.866882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:57.873490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:57.874029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:57.874694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:57.875413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:57.876001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:57.876596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:58.007693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:58.015197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-beeac6a3-ea9e-4a2c-b5a5-c2d0e48cbc09 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.152822 Oct 09 01:00:58.015780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 313/620] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:57 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:00:59.039547 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:00:59.040276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:00:59.046337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:59.046810 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:59.047405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:59.048146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:00:59.048857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:00:59.049295 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:00:59.069770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:00:59.186850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:00:59.200387 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cfa50978-e194-4d41-ad46-0583e9b219e6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.165923 Oct 09 01:00:59.200962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 308/621] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:00:59 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:00.223439 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:00.224065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:00.226132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:00.231511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:00.232016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:00.232446 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:00.233127 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:00.233573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:00.233987 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:00.360712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:00.368719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-63bf1d42-1a8e-4ddb-89ab-9f35635e14ee tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.149227 Oct 09 01:01:00.369280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 314/622] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:00 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:01.392069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:01.392618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:01.400201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:01.400675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:01.401101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:01.402046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:01.402490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:01.402907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:01.535353 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:01.543633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9b068fa2-0ec5-4e4e-bc50-4796cad6d15b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.155417 Oct 09 01:01:01.544193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 309/623] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:01 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:01.725492 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-8ea9805d-fab3-4caf-8a80-f51e605f62c7 req-7c2a5e80-e124-4269-a247-bb288bc2a41a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e0d9a7c4-b33c-42c1-9042-dfd357bc3932", "name": "network-vif-plugged", "status": "completed", "tag": "bf1ecde5-a035-439a-96b4-f7a4880b70e4"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:01.735577 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-8ea9805d-fab3-4caf-8a80-f51e605f62c7 req-7c2a5e80-e124-4269-a247-bb288bc2a41a service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:01.736079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-8ea9805d-fab3-4caf-8a80-f51e605f62c7 req-7c2a5e80-e124-4269-a247-bb288bc2a41a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:01.736570 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-8ea9805d-fab3-4caf-8a80-f51e605f62c7 req-7c2a5e80-e124-4269-a247-bb288bc2a41a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:01.748673 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-8ea9805d-fab3-4caf-8a80-f51e605f62c7 req-7c2a5e80-e124-4269-a247-bb288bc2a41a service nova] Creating event network-vif-plugged:bf1ecde5-a035-439a-96b4-f7a4880b70e4 for instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 on hv-cinder-maste Oct 09 01:01:01.754108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-8ea9805d-fab3-4caf-8a80-f51e605f62c7 req-7c2a5e80-e124-4269-a247-bb288bc2a41a service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032364 Oct 09 01:01:01.754777 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 315/624] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:02.566259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:02.566886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:02.573253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:02.573730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:02.574170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:02.574854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:02.575305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:02.575756 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:02.704227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:02.712246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d264d7f4-c462-4ac8-a2c9-1b8a48b21e1f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.149883 Oct 09 01:01:02.712806 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 310/625] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:02 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:03.735471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:03.736119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:03.742203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:03.742684 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:03.743128 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:03.743836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:03.744292 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:03.744718 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:03.876595 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:03.885881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-60540330-e08b-49b5-b1b8-7c9645ffb0d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1365 microversion: 2.1 time: 0.155231 Oct 09 01:01:03.886460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 316/626] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:03 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1365 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:04.909401 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:04.910002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:04.912290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:04.912944 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:04.919001 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:04.919481 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:04.919937 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:04.920623 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:04.921076 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:04.921499 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:05.049141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:05.057258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-60ad0250-c1a3-4d1f-b16b-c2ac1fa0663b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1556 microversion: 2.1 time: 0.151693 Oct 09 01:01:05.057816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 311/627] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:04 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1556 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:05.079173 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:05.079711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:05.081597 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:05.087205 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:05.087719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:05.088161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:05.088847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:05.089298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:05.089723 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:05.165201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ef01a383-e3ff-48a0-8f6b-dbb858330b4b tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.089837 Oct 09 01:01:05.165791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 317/628] 10.222.0.33 () {58 vars in 1275 bytes} [Sun Oct 9 01:01:05 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-interface => generated 285 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:06.124505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:06.125063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:06.131765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:06.132251 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:06.132691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:06.133378 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:06.133878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:06.134289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:06.217920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lazy-loading 'services' on Instance uuid 50224067-d3e0-4567-bc63-31ace3a211c6 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:06.302004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b2041a30-fe94-4dfe-b2a1-140b7a269d02 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments/af3f6b36-1fc1-4dde-9076-c1ec90e38994" status: 202 len: 0 microversion: 2.1 time: 0.181545 Oct 09 01:01:06.302799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 312/629] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:01:06 2022] DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments/af3f6b36-1fc1-4dde-9076-c1ec90e38994 => generated 0 bytes in 183 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:01:06.312265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:06.315773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:06.322575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-caace48b-9ee0-4677-acf3-ebadd16e251b req-66704984-3a01-4c28-b88f-51b1e24d73cf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e0d9a7c4-b33c-42c1-9042-dfd357bc3932", "tag": "bf1ecde5-a035-439a-96b4-f7a4880b70e4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:06.323457 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:06.324208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:06.324919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:06.325854 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:06.326458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:06.327034 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:06.333940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-caace48b-9ee0-4677-acf3-ebadd16e251b req-66704984-3a01-4c28-b88f-51b1e24d73cf service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:06.334473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-caace48b-9ee0-4677-acf3-ebadd16e251b req-66704984-3a01-4c28-b88f-51b1e24d73cf service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:06.334844 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-caace48b-9ee0-4677-acf3-ebadd16e251b req-66704984-3a01-4c28-b88f-51b1e24d73cf service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:06.346285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-caace48b-9ee0-4677-acf3-ebadd16e251b req-66704984-3a01-4c28-b88f-51b1e24d73cf service nova] Creating event network-changed:bf1ecde5-a035-439a-96b4-f7a4880b70e4 for instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 on hv-cinder-maste Oct 09 01:01:06.351097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-caace48b-9ee0-4677-acf3-ebadd16e251b req-66704984-3a01-4c28-b88f-51b1e24d73cf service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033035 Oct 09 01:01:06.351735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 313/630] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:01:06.360073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-207fd394-0cf3-4b89-8831-f15475bc559b tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052647 Oct 09 01:01:06.360642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 318/631] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:06 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:07.373789 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:07.376277 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:07.383023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:07.383495 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:07.384026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:07.384724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:07.385354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:07.385937 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:07.420435 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2b2eddca-ac44-427e-b606-ae2a690242e6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051427 Oct 09 01:01:07.420993 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 314/632] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:07 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:08.432463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:08.434624 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:08.441147 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:08.441622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:08.442057 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:08.442731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:08.443178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:08.443618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:08.472304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dcf85607-8996-49d4-be3b-91b78b165c6e tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043728 Oct 09 01:01:08.472860 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 319/633] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:08 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:01:08.548493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:08.549008 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:08.555300 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:08.555764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:08.556201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:08.556878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:08.557326 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:08.557747 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:08.582614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:01:08.660130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2f3faef9-15ac-4cc6-9360-be8d5a1d995f tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 204 len: 0 microversion: 2.1 time: 0.114063 Oct 09 01:01:08.660815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 315/634] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:08 2022] DELETE /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:01:08.668659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:08.669271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:08.675113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:08.675614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:08.676056 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:08.676741 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:08.677193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:08.677615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:08.809121 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:08.817311 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1573fab4-3cee-4833-9fc2-a0d5f9df23c0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1688 microversion: 2.1 time: 0.152468 Oct 09 01:01:08.817872 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 320/635] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:08 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1688 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:09.840752 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:09.841441 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:09.848397 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:09.848882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:09.849324 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:09.850019 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:09.850475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:09.850903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:10.033407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:10.041739 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-538fbbea-9fae-4d61-ac16-a80e88fb9942 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1688 microversion: 2.1 time: 0.205220 Oct 09 01:01:10.042300 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 316/636] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:09 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1688 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:11.064736 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:11.065256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:11.072109 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:11.072596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:11.073057 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:11.073742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:11.074191 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:11.074612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:11.200314 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:11.209404 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2c18584e-9db7-43a1-877b-671fefbba4e4 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1688 microversion: 2.1 time: 0.148520 Oct 09 01:01:11.209962 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 321/637] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:11 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1688 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:12.232936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:12.233544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:12.235965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=63,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:12.241758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:12.242241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:12.242675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:12.243361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:12.243840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:12.244265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:12.372078 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:12.379724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f70b3017-3641-4859-b5c5-59eb22dc24ca tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1688 microversion: 2.1 time: 0.150663 Oct 09 01:01:12.380284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 317/638] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:12 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1688 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:13.313611 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-4fe37bfb-1d20-4c8e-8187-7501679186a7 req-0ff5d001-2914-47ff-b7ac-48b0269a82e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "50224067-d3e0-4567-bc63-31ace3a211c6", "name": "network-vif-deleted", "tag": "af11de25-9b71-43af-8ce0-051a32ec945c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:13.324052 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-4fe37bfb-1d20-4c8e-8187-7501679186a7 req-0ff5d001-2914-47ff-b7ac-48b0269a82e6 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:13.324616 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-4fe37bfb-1d20-4c8e-8187-7501679186a7 req-0ff5d001-2914-47ff-b7ac-48b0269a82e6 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:13.325113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-4fe37bfb-1d20-4c8e-8187-7501679186a7 req-0ff5d001-2914-47ff-b7ac-48b0269a82e6 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:13.336514 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-4fe37bfb-1d20-4c8e-8187-7501679186a7 req-0ff5d001-2914-47ff-b7ac-48b0269a82e6 service nova] Creating event network-vif-deleted:af11de25-9b71-43af-8ce0-051a32ec945c for instance 50224067-d3e0-4567-bc63-31ace3a211c6 on hv-cinder-maste Oct 09 01:01:13.341684 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-4fe37bfb-1d20-4c8e-8187-7501679186a7 req-0ff5d001-2914-47ff-b7ac-48b0269a82e6 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032038 Oct 09 01:01:13.342468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 322/639] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:13.403911 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:13.405130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:13.413081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:13.413915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:13.414356 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:13.415124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:13.415568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:13.416091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:13.455250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:13.456052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:13.456730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:13.457182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lazy-loading 'fault' on Instance uuid 50224067-d3e0-4567-bc63-31ace3a211c6 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:13.524084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:13.535908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-584f3107-25fa-48cd-b976-9671f295b051 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 200 len: 1336 microversion: 2.1 time: 0.136057 Oct 09 01:01:13.536800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 318/640] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:13 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 1336 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:14.561223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:14.562042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: 50224067-d3e0-4567-bc63-31ace3a211c6] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:14.568195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:14.568681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:14.569122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:14.569810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:14.570265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:14.570690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:14.585238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] HTTP exception thrown: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 could not be found. Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Request method failure captured: Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 HTTP/1.1 Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwpCa7qTsGCbgYzI3hTp0XV4aX7UY9kYcqRmcLmnNuVu0pwn81JmE0Q7Hu-M48YX0Ivzl80hJOivFomkM56cwHMuXcUttfPVYuxDmo1TXJ7o2oEOERlIn_KBndSYXeiVrB5ti0ZjjXNGyRZ2bEWEYWMfg6VZDBo93QOHqW60b7MagU Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:01:14.586841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 438446d1e6b3402bb18b547c089391f8 Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 could not be found. Oct 09 01:01:14.588429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '50224067-d3e0-4567-bc63-31ace3a211c6'} Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 could not be found. Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:14.589315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 could not be found. Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 could not be found. Oct 09 01:01:14.590796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:14.592153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Returning 404 to user: Instance 50224067-d3e0-4567-bc63-31ace3a211c6 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:14.592153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e6f77def-835a-4e83-979b-ea28bc5e0c54 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6" status: 404 len: 111 microversion: 2.1 time: 0.031761 Oct 09 01:01:14.592153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 323/641] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:14 2022] GET /compute/v2.1/servers/50224067-d3e0-4567-bc63-31ace3a211c6 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:14.596823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-29618bf6-ce60-4db2-984c-b5737cd95a1d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '.version_select at 0x7f7268a4c3a0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:14.616130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-29618bf6-ce60-4db2-984c-b5737cd95a1d tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2109480898" status: 202 len: 0 microversion: 2.1 time: 0.022965 Oct 09 01:01:14.616637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 319/642] 10.222.0.33 () {58 vars in 1221 bytes} [Sun Oct 9 01:01:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2109480898 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:01:17.464887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:17.465389 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:17.471748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:17.472222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:17.472658 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:17.473325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:17.473767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:17.474182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:17.478062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:17.501176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:01:17.574037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-732c90dd-41aa-4bd7-acd5-e8c841893168 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "DELETE /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 204 len: 0 microversion: 2.1 time: 0.112505 Oct 09 01:01:17.574944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 324/643] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:17 2022] DELETE /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:01:17.584629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:17.585243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:17.591785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:17.592278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:17.592721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:17.593420 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:17.593914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:17.594349 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:17.724595 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:17.732204 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6aca6fb6-3044-46a6-99f2-322d9d533127 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1507 microversion: 2.1 time: 0.152180 Oct 09 01:01:17.732785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 320/644] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:17 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1507 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:18.756953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:18.757637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:18.763968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:18.764442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:18.764870 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:18.765543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:18.765984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:18.766395 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:18.893644 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:18.894873 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:18.899002 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:18.901704 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:18.902185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:18.902620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:18.903312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:18.903793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:18.904218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:18.906883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-33ed1a4b-fe81-4a7f-b986-a5cc40fd24ae tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1507 microversion: 2.1 time: 0.154621 Oct 09 01:01:18.907435 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 325/645] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:18 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1507 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:18.935500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:01:19.009406 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-398aff76-9a75-4414-b2c9-c5622f47fd43 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "DELETE /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 204 len: 0 microversion: 2.1 time: 0.119375 Oct 09 01:01:19.009903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 321/646] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:18 2022] DELETE /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 09 01:01:19.017980 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:19.018638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:19.024568 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:19.025046 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:19.025474 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:19.026403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:19.026849 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:19.027265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:19.155032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:19.162580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1126dd60-3185-4fe1-8d4f-32390cd68767 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1578 microversion: 2.1 time: 0.148373 Oct 09 01:01:19.163135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 326/647] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:19 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1578 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:19.932203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:19.932859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:19.939472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:19.939991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:19.940430 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:19.941121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:19.941569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:19.941987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:19.960411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:20.076386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:20.084637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6415e429-f012-4b98-8e50-27f47bd50d17 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1507 microversion: 2.1 time: 0.156776 Oct 09 01:01:20.085192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 322/648] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:19 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1507 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:20.174301 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:20.174793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:20.180859 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:20.181067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:20.181504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:20.182188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:20.182638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:20.183057 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:20.185645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:20.186253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:20.192629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:20.193108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:20.193545 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:20.194225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:20.194674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:20.195095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:20.208519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-11c7f1f7-41d3-48df-8419-088399a30799 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/ips" status: 200 len: 146 microversion: 2.1 time: 0.090125 Oct 09 01:01:20.209079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 327/649] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 01:01:20 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/ips => generated 146 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:20.330511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:20.339116 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1d766a50-f840-4913-b670-962781e19ce8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1578 microversion: 2.1 time: 0.157187 Oct 09 01:01:20.340144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 323/650] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:20 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1578 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:21.107095 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:21.107724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:21.114174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:21.114644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:21.115076 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:21.115780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:21.116226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:21.116644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:21.243627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:21.251184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7b6c1799-5a37-4679-b337-f4d9302e42d1 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1507 microversion: 2.1 time: 0.147858 Oct 09 01:01:21.251764 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 328/651] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:21 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1507 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:21.362329 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:21.362858 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:21.369262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:21.369744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:21.370180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:21.370871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:21.371327 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:21.371780 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:21.500081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:21.508241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c8897a0e-3db3-41c3-89b2-860fbda43dd4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1578 microversion: 2.1 time: 0.149782 Oct 09 01:01:21.508804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 324/652] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:21 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1578 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:21.543186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f16e84ec-8a34-4d2e-a8ad-57ac07475fa6"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:21.545638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:21.551413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:21.551927 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:21.552356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:21.553029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:21.553475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:21.553893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:21.967855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5722f0cb-1a5e-4a40-a214-64f5467b39d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.428183 Oct 09 01:01:21.974765 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 329/653] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:01:21 2022] POST /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments => generated 194 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:22.273361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:22.273992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:22.280690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:22.281217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:22.281718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:22.282348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:22.282806 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:22.283228 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:22.424074 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:22.432645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-78100ec2-4e07-44ca-885e-82ac732957c8 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 200 len: 1507 microversion: 2.1 time: 0.163149 Oct 09 01:01:22.433305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 325/654] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:22 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 1507 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:22.532275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:22.533255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:22.542211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:22.542789 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:22.543316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:22.544160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:22.546081 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:22.546689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:22.668138 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-762e9a5e-beae-4b74-8f0b-d377d6ed1652 req-a127385e-b3ef-4565-8916-fdfccaf0ca95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b86736a3-d8f2-450d-a367-4d2ba4edd923", "name": "network-vif-deleted", "tag": "8fe52a88-9c39-4ea8-9d6f-ca693b1da44c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:22.679024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-762e9a5e-beae-4b74-8f0b-d377d6ed1652 req-a127385e-b3ef-4565-8916-fdfccaf0ca95 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:22.679403 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-762e9a5e-beae-4b74-8f0b-d377d6ed1652 req-a127385e-b3ef-4565-8916-fdfccaf0ca95 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:22.679861 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-762e9a5e-beae-4b74-8f0b-d377d6ed1652 req-a127385e-b3ef-4565-8916-fdfccaf0ca95 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:22.693691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-762e9a5e-beae-4b74-8f0b-d377d6ed1652 req-a127385e-b3ef-4565-8916-fdfccaf0ca95 service nova] Creating event network-vif-deleted:8fe52a88-9c39-4ea8-9d6f-ca693b1da44c for instance b86736a3-d8f2-450d-a367-4d2ba4edd923 on hv-cinder-maste Oct 09 01:01:22.699480 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-762e9a5e-beae-4b74-8f0b-d377d6ed1652 req-a127385e-b3ef-4565-8916-fdfccaf0ca95 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.147539 Oct 09 01:01:22.701393 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 326/655] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:22.777504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:22.786582 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-abc1cd62-04c3-4566-8b65-a047ac9e54d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1578 microversion: 2.1 time: 0.257888 Oct 09 01:01:22.787174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 330/656] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:22 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1578 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:23.460311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:23.461030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: b86736a3-d8f2-450d-a367-4d2ba4edd923] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:23.467344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.467853 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.468295 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.468992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.469450 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.469878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.484415 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] HTTP exception thrown: Instance b86736a3-d8f2-450d-a367-4d2ba4edd923 could not be found. Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Request method failure captured: Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 HTTP/1.1 Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwpCa7qTsGCbgYzI3hTp0XV4aX7UY9kYcqRmcLmnNuVu0pwn81JmE0Q7Hu-M48YX0Ivzl80hJOivFomkM56cwHMuXcUttfPVYuxDmo1TXJ7o2oEOERlIn_KBndSYXeiVrB5ti0ZjjXNGyRZ2bEWEYWMfg6VZDBo93QOHqW60b7MagU Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:01:23.486440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 438446d1e6b3402bb18b547c089391f8 Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance b86736a3-d8f2-450d-a367-4d2ba4edd923 could not be found. Oct 09 01:01:23.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'b86736a3-d8f2-450d-a367-4d2ba4edd923'} Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b86736a3-d8f2-450d-a367-4d2ba4edd923 could not be found. Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:23.488907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b86736a3-d8f2-450d-a367-4d2ba4edd923 could not be found. Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b86736a3-d8f2-450d-a367-4d2ba4edd923 could not be found. Oct 09 01:01:23.490900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:23.493005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Returning 404 to user: Instance b86736a3-d8f2-450d-a367-4d2ba4edd923 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:23.493005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-be661643-d350-4549-8262-746df79f8762 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923" status: 404 len: 111 microversion: 2.1 time: 0.031907 Oct 09 01:01:23.493005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 327/657] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:23 2022] GET /compute/v2.1/servers/b86736a3-d8f2-450d-a367-4d2ba4edd923 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 01:01:23.497281 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:23.497817 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:23.504535 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.505592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.506029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.506742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.507198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.507643 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.536498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:01:23.616862 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2f3b8ccf-5736-4d15-9526-a5e0c749fad6 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "DELETE /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 204 len: 0 microversion: 2.1 time: 0.123219 Oct 09 01:01:23.621054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 331/658] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:23 2022] DELETE /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:01:23.628217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:23.629202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:23.638430 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.639068 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.639620 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.640431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.640975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.641658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.779576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:23.787430 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-45a3a9da-0565-47db-a2a5-879e76e9a31c tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.164443 Oct 09 01:01:23.788018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 328/659] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:23 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:23.811922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:23.812745 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:23.819655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.820262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.820694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.821381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:23.821827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:23.822246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:23.885680 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:23.893529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ece98d40-bc57-4dcd-b8e8-e7cf9efe86b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1225 microversion: 2.1 time: 0.085782 Oct 09 01:01:23.894167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 332/660] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:23 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1225 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:24.714907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-d6710782-54fc-4ab1-b707-6904bcca5915 req-f9a38a05-bffc-424a-91db-ce8505531be1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "841a7957-ee73-4765-a850-406b1e720612", "name": "network-vif-deleted", "tag": "c0379d91-6e4b-4a6c-895c-fc71a455dbb1"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:24.725417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-d6710782-54fc-4ab1-b707-6904bcca5915 req-f9a38a05-bffc-424a-91db-ce8505531be1 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:24.725882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-d6710782-54fc-4ab1-b707-6904bcca5915 req-f9a38a05-bffc-424a-91db-ce8505531be1 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:24.726313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-d6710782-54fc-4ab1-b707-6904bcca5915 req-f9a38a05-bffc-424a-91db-ce8505531be1 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:24.738467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-d6710782-54fc-4ab1-b707-6904bcca5915 req-f9a38a05-bffc-424a-91db-ce8505531be1 service nova] Creating event network-vif-deleted:c0379d91-6e4b-4a6c-895c-fc71a455dbb1 for instance 841a7957-ee73-4765-a850-406b1e720612 on hv-cinder-maste Oct 09 01:01:24.743988 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-d6710782-54fc-4ab1-b707-6904bcca5915 req-f9a38a05-bffc-424a-91db-ce8505531be1 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032618 Oct 09 01:01:24.744558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 329/661] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:24.810086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:24.810687 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:24.816711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:24.817214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:24.817645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:24.818317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:24.818763 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:24.819178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:24.918219 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:24.919013 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:24.925153 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:24.925639 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:24.926081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:24.926783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:24.927243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:24.927773 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:24.953597 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:24.962048 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bf96f08b-7fdd-4d73-ac73-18fad49e11f7 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.155639 Oct 09 01:01:24.962534 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 333/662] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:24 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:24.990208 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:24.997984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-29e91663-d198-4b55-b790-bbf87dbc4be1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1225 microversion: 2.1 time: 0.083977 Oct 09 01:01:24.998541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 330/663] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:24 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1225 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:25.984566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:25.985166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:25.991891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:25.992379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:25.992814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:25.993500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:25.993953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:25.994374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:26.020116 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:26.020808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:26.027582 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:26.028567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:26.028703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:26.029371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:26.029920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:26.030416 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:26.108587 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:26.118253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ebecc45d-1a1b-4904-989c-f7d6c81020b2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1225 microversion: 2.1 time: 0.101903 Oct 09 01:01:26.128104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 331/664] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:26 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:26.134532 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:26.145575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-16077d77-7197-4276-a691-9ccb1eb76bf0 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.164832 Oct 09 01:01:26.146185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 334/665] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:25 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:27.140451 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:27.141061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:27.147921 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:27.148198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:27.148637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:27.149330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:27.149783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:27.150209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:27.170619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:27.173749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:27.180433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:27.181022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:27.181685 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:27.182588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:27.183503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:27.184160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:27.186230 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:27.186718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:27.187157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:27.187652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid 841a7957-ee73-4765-a850-406b1e720612 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:27.234704 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:27.243134 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-435316c2-542b-4dc1-9e08-ad7cdbd72322 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 200 len: 1230 microversion: 2.1 time: 0.106495 Oct 09 01:01:27.243729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 332/666] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:27 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 1230 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:27.423333 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:27.431445 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0f418e33-3062-47e2-a7da-84ee049a8e39 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1520 microversion: 2.1 time: 0.265908 Oct 09 01:01:27.432058 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 335/667] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:27 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1520 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:27.941235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-3c9130e7-49e7-4fd7-9d1d-4a077e54a8bd req-ef16823a-a314-4cbb-b266-5f07d453541a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1b50472-3065-47ec-85b8-f52e36a9374c", "name": "network-vif-deleted", "tag": "c41ba939-ec56-4a6f-86b0-047036daa405"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:27.951129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-3c9130e7-49e7-4fd7-9d1d-4a077e54a8bd req-ef16823a-a314-4cbb-b266-5f07d453541a service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:27.951616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-3c9130e7-49e7-4fd7-9d1d-4a077e54a8bd req-ef16823a-a314-4cbb-b266-5f07d453541a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:27.952053 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-3c9130e7-49e7-4fd7-9d1d-4a077e54a8bd req-ef16823a-a314-4cbb-b266-5f07d453541a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:27.962880 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-3c9130e7-49e7-4fd7-9d1d-4a077e54a8bd req-ef16823a-a314-4cbb-b266-5f07d453541a service nova] Creating event network-vif-deleted:c41ba939-ec56-4a6f-86b0-047036daa405 for instance a1b50472-3065-47ec-85b8-f52e36a9374c on hv-cinder-maste Oct 09 01:01:27.967614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-3c9130e7-49e7-4fd7-9d1d-4a077e54a8bd req-ef16823a-a314-4cbb-b266-5f07d453541a service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029935 Oct 09 01:01:27.968210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 333/668] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:28.266312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:28.266942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:28.275502 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.276639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.277076 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.277773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.278216 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.278637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.297604 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] HTTP exception thrown: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Request method failure captured: Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 HTTP/1.1 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhyCzkbPt7AN3CTBkhz3r1POANnV9yaKeP2DZb2JPHpowTWCg7kviO4vrxDOli09Ug3OcMM-tmCxW5VH6AozJn5asWgto319ZeOwP7b0SF9yuk0cpFS3940Fk36BSylO6KrJAXuwZhEGiJ21v57PhWgfYWS3DrBgWsy-F7TQs1q5YO4 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:01:28.299254 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-TestVolumeBootPattern-121914356 Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 04bc22d558c2466ba43a456d08bdd6bd Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:01:28.300581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '841a7957-ee73-4765-a850-406b1e720612'} Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:28.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:01:28.303131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:28.304230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Returning 404 to user: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:28.304230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-37487541-d783-4108-aa3b-b8e9e02cb17d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 404 len: 111 microversion: 2.1 time: 0.038309 Oct 09 01:01:28.304230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 336/669] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:28 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:28.417695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-564144669", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1094622065", "security_groups": [{"name": "tempest-secgroup-smoke-606774372"}], "block_device_mapping_v2": [{"uuid": "b9013137-781e-4610-982f-a64062ab5a38", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "51f66c80-6e94-4ac9-ad00-6dd9246260d1"}]}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:28.454872 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:28.455494 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:28.461497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.461967 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.462405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.463131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.463536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.463983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.467927 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:28.499089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.499575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.500043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.500489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lazy-loading 'fault' on Instance uuid a1b50472-3065-47ec-85b8-f52e36a9374c {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:28.531946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] validate_networks() for [('51f66c80-6e94-4ac9-ad00-6dd9246260d1', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:01:28.540933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '438446d1e6b3402bb18b547c089391f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f68941040b3049bd9c62e2fc061d94c1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:28.548784 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bfca362c-d7e2-4a3f-b389-370b319568a5 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 200 len: 1311 microversion: 2.1 time: 0.097754 Oct 09 01:01:28.549341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 337/670] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:28 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 1311 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:28.958754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:01:28.959172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:01:28.959670 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:01:28.960102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:01:28.960507 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:01:28.969284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.969702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.970135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.970854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:28.971305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:28.971758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:28.988466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:01:28.992414 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for user 04bc22d558c2466ba43a456d08bdd6bd and project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:01:29.050121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:01:29.052442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b9013137-781e-4610-982f-a64062ab5a38',volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:01:29.190318 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6cd1b6bf-2dcd-46ee-b210-82da2381157f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.775459 Oct 09 01:01:29.197645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 334/671] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:01:28 2022] POST /compute/v2.1/servers => generated 405 bytes in 779 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:01:29.203922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6de60587-ee8f-4b75-a07a-0dfa23818b9a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:29.204625 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6de60587-ee8f-4b75-a07a-0dfa23818b9a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:29.262808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6de60587-ee8f-4b75-a07a-0dfa23818b9a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:29.270713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6de60587-ee8f-4b75-a07a-0dfa23818b9a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1094 microversion: 2.1 time: 0.071060 Oct 09 01:01:29.271337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 338/672] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:29 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1094 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:29.519787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:29.522597 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:29.529113 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:29.529598 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:29.530036 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:29.530719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:29.531166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:29.531615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:29.561460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-08c0077e-59bf-4c1b-a8a4-e1c13ebcd776 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046197 Oct 09 01:01:29.562293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 335/673] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:29 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:29.573645 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:29.574087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:29.574259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:29.574874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] [instance: a1b50472-3065-47ec-85b8-f52e36a9374c] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:29.580588 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:29.581069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:29.581507 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:29.582187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:29.582332 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:29.583190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:29.583632 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:29.583778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:29.584232 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:29.585012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:29.585536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:29.586031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:29.601690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] HTTP exception thrown: Instance a1b50472-3065-47ec-85b8-f52e36a9374c could not be found. Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Request method failure captured: Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c HTTP/1.1 Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwpCa7qTsGCbgYzI3hTp0XV4aX7UY9kYcqRmcLmnNuVu0pwn81JmE0Q7Hu-M48YX0Ivzl80hJOivFomkM56cwHMuXcUttfPVYuxDmo1TXJ7o2oEOERlIn_KBndSYXeiVrB5ti0ZjjXNGyRZ2bEWEYWMfg6VZDBo93QOHqW60b7MagU Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:01:29.603541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f68941040b3049bd9c62e2fc061d94c1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: f68941040b3049bd9c62e2fc061d94c1 Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1939425555 Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 438446d1e6b3402bb18b547c089391f8 Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1939425555-project Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance a1b50472-3065-47ec-85b8-f52e36a9374c could not be found. Oct 09 01:01:29.605045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'a1b50472-3065-47ec-85b8-f52e36a9374c'} Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1b50472-3065-47ec-85b8-f52e36a9374c could not be found. Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:29.605831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1b50472-3065-47ec-85b8-f52e36a9374c could not be found. Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1b50472-3065-47ec-85b8-f52e36a9374c could not be found. Oct 09 01:01:29.607163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:29.609105 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] Returning 404 to user: Instance a1b50472-3065-47ec-85b8-f52e36a9374c could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:29.610104 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3192d495-f931-492b-90a9-2ca35cf59e83 tempest-ServerRescueNegativeTestJSON-1939425555 tempest-ServerRescueNegativeTestJSON-1939425555-project] 10.222.0.33 "GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c" status: 404 len: 111 microversion: 2.1 time: 0.041124 Oct 09 01:01:29.611131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 339/674] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:29 2022] GET /compute/v2.1/servers/a1b50472-3065-47ec-85b8-f52e36a9374c => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:29.616121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-eeb82cfd-1f95-4f73-bb42-235116192c3e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/f16e84ec-8a34-4d2e-a8ad-57ac07475fa6" status: 200 len: 194 microversion: 2.1 time: 0.046448 Oct 09 01:01:29.617027 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 336/675] 10.222.0.33 () {58 vars in 1413 bytes} [Sun Oct 9 01:01:29 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/f16e84ec-8a34-4d2e-a8ad-57ac07475fa6 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:01:30.293611 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:30.294179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:30.295956 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:30.301682 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:30.302162 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:30.302592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:30.303271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:30.303751 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:30.304174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:30.386793 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:30.394552 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a407e81e-a897-463b-b925-167cad0b66ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1192 microversion: 2.1 time: 0.104836 Oct 09 01:01:30.409699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 340/676] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:30 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1192 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:31.004896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "315fd8d2-9660-4848-a58f-565b28e6e3ce"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:31.007278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:31.014353 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:31.015239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:31.016101 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:31.017090 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:31.017543 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:31.018018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:31.332310 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d38097b5-8df2-4e0b-abd8-a5c64029969f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "POST /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.331129 Oct 09 01:01:31.333802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 337/677] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:01:30 2022] POST /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments => generated 194 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:31.417860 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:31.418465 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:31.424882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:31.425565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:31.426228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:31.427558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:31.428181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:31.428763 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:31.518034 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:31.526480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-042a567e-80de-4978-aa74-f7dbed5bc85b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1210 microversion: 2.1 time: 0.111495 Oct 09 01:01:31.527054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 341/678] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:31 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1210 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:32.550850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:32.552419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:32.568609 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:32.569930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:32.570854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:32.573608 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:32.575320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:32.576577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:32.863292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:32.871750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7c986fb5-4117-4b6b-add5-1194121d823a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.325208 Oct 09 01:01:32.872466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 338/679] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:32 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:32.940773 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-2ca9ebab-bcd4-42ff-ab2b-e52c7064076a req-4c66611a-fab7-4825-8186-7d31650f229d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "edbfc046-964c-46ac-9d30-5198c846e9b5", "tag": "92161e7f-b2e3-4be5-9ff8-073cd2234c8e"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:32.955407 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-2ca9ebab-bcd4-42ff-ab2b-e52c7064076a req-4c66611a-fab7-4825-8186-7d31650f229d service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:32.956309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-2ca9ebab-bcd4-42ff-ab2b-e52c7064076a req-4c66611a-fab7-4825-8186-7d31650f229d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:32.957012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-2ca9ebab-bcd4-42ff-ab2b-e52c7064076a req-4c66611a-fab7-4825-8186-7d31650f229d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:32.969591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-2ca9ebab-bcd4-42ff-ab2b-e52c7064076a req-4c66611a-fab7-4825-8186-7d31650f229d service nova] Creating event network-changed:92161e7f-b2e3-4be5-9ff8-073cd2234c8e for instance edbfc046-964c-46ac-9d30-5198c846e9b5 on hv-cinder-maste Oct 09 01:01:32.985775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-2ca9ebab-bcd4-42ff-ab2b-e52c7064076a req-4c66611a-fab7-4825-8186-7d31650f229d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049440 Oct 09 01:01:32.987734 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 342/680] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:33.894519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:33.895842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:33.902788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:33.903280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:33.903746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:33.904460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:33.904917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:33.905342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:34.221813 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:34.233050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-503a6027-fee1-4b8d-a72e-4b879ac89ff3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.342355 Oct 09 01:01:34.249610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 339/681] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:33 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:35.258024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:35.258722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:35.264804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:35.265477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:35.265717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:35.266480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:35.266932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:35.267357 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:35.405900 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:35.413725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e8869d19-2f26-44df-a842-3afc31ee753d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.160964 Oct 09 01:01:35.428013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 343/682] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:35 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:36.436919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:36.437632 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:36.444407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:36.444887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:36.445319 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:36.446035 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:36.446486 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:36.447202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:36.599272 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:36.603238 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:36.608347 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-95ff0a02-ed06-4fb6-8652-75cd6183816a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.175126 Oct 09 01:01:36.609165 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 340/683] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:36 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:37.630493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:37.631769 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:37.638930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:37.639501 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:37.640157 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:37.641174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:37.641828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:37.642450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:37.803850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:37.812367 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-022aefe0-31a3-47f0-bc5f-cffb2c4820e4 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.185665 Oct 09 01:01:37.823191 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 344/684] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:37 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:38.838936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:38.840294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:38.848765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:38.849438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:38.850058 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:38.850777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:38.851238 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:38.852357 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:38.981289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:39.005493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:39.020448 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:39.023396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:39.030709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:39.031445 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:39.031898 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:39.032594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:39.033099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:39.033518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:39.067860 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2cbcc5ae-6236-4262-87d4-0d81fb1738f4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.052106 Oct 09 01:01:39.068434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 345/685] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:39 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments => generated 371 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:39.100679 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:39.109948 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7edf1863-3316-495b-864c-6647f00d106f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.277524 Oct 09 01:01:39.111339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 341/686] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:38 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:39.223870 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:39.224621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:39.236826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:39.237594 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:39.239027 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:39.240759 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:39.241230 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:39.241919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:39.270103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:39.284221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5dbdd36b-59ba-45bf-9c8e-0053a2b99c20 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/f16e84ec-8a34-4d2e-a8ad-57ac07475fa6" status: 200 len: 194 microversion: 2.1 time: 0.066151 Oct 09 01:01:39.284846 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 342/687] 10.222.0.33 () {58 vars in 1413 bytes} [Sun Oct 9 01:01:39 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/f16e84ec-8a34-4d2e-a8ad-57ac07475fa6 => generated 194 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:39.295715 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:39.296189 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:39.302771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:39.303266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:39.303727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:39.304436 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:39.304894 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:39.305319 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:39.352137 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c5f2eceb-b23e-4203-bf83-939e3aaf1b51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'create', calling method: .version_select at 0x7f7268a4c940>, body: {"keypair": {"name": "tempest-keypair-1083022739", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCDRgaXSo3+rrTblCVrwJFdG9ndkhCEDlbvIZYrXc3PRqMNt8/1m0HBuc9qLeZ12Os3+xTMVjyTyobYFBEAS2ZjdiYW+vc+M5JhwVFuru8gTY+pNEKArzZrfkrOkmAnD0w=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:39.368808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-c5f2eceb-b23e-4203-bf83-939e3aaf1b51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for project 410212da4a794c07b30ed22cc5b7e760. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:01:39.373263 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-c5f2eceb-b23e-4203-bf83-939e3aaf1b51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for user 56b5d764f0044358ac594a2edd6f4c0f and project 410212da4a794c07b30ed22cc5b7e760. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:01:39.400516 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c5f2eceb-b23e-4203-bf83-939e3aaf1b51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.182336 Oct 09 01:01:39.400516 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 346/688] 10.222.0.33 () {62 vars in 1200 bytes} [Sun Oct 9 01:01:39 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 184 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 09 01:01:39.406477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lazy-loading 'services' on Instance uuid bf1a22b3-a821-421e-8d9e-93190101a78d {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:39.502088 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f5431837-21a3-4031-bca9-9e0a32e21114 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/f16e84ec-8a34-4d2e-a8ad-57ac07475fa6" status: 202 len: 0 microversion: 2.1 time: 0.210363 Oct 09 01:01:39.503476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 343/689] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:01:39 2022] DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/f16e84ec-8a34-4d2e-a8ad-57ac07475fa6 => generated 0 bytes in 212 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 01:01:40.135099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:40.135940 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:40.144444 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:40.145485 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:40.146312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:40.147283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:40.148073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:40.148664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:40.295649 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:40.305077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cea2ba00-013a-4934-bab3-661f891f8cad tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.174319 Oct 09 01:01:40.305963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 347/690] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:40 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:40.822898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1930257025", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d442b64f-563d-410a-8314-ec0e04e8ccd0"}], "security_groups": [{"name": "tempest-securitygroup--513233236"}], "key_name": "tempest-keypair-1083022739", "user_data": "}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:40.955064 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] validate_networks() for [('d442b64f-563d-410a-8314-ec0e04e8ccd0', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:01:41.329364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:41.330781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:41.338639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.339190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.339819 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.340829 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.341471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.341898 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.349076 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:01:41.349671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:01:41.349997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:01:41.350405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:01:41.350864 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:01:41.364919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.365574 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.366018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.366745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.367241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.367654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.384519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for project 410212da4a794c07b30ed22cc5b7e760. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:01:41.388644 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for user 56b5d764f0044358ac594a2edd6f4c0f and project 410212da4a794c07b30ed22cc5b7e760. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:01:41.402895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:01:41.405417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:01:41.455940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d72e96ff-26c0-4976-860d-883cdffb4f4c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.637633 Oct 09 01:01:41.459642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 344/691] 10.222.0.33 () {62 vars in 1188 bytes} [Sun Oct 9 01:01:40 2022] POST /compute/v2.1/servers => generated 401 bytes in 640 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 09 01:01:41.468766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7e21d420-cf18-4229-a175-c5a53a035181 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:41.469852 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7e21d420-cf18-4229-a175-c5a53a035181 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:41.476543 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:41.508265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:41.516494 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-466ed4dc-04cf-4499-af0c-3602b9fe272b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.191868 Oct 09 01:01:41.517384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 348/692] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:41 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:41.542478 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7e21d420-cf18-4229-a175-c5a53a035181 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:41.551201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7e21d420-cf18-4229-a175-c5a53a035181 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:41.552638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7e21d420-cf18-4229-a175-c5a53a035181 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:41.553750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7e21d420-cf18-4229-a175-c5a53a035181 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1282 microversion: 2.20 time: 0.089282 Oct 09 01:01:41.554311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 345/693] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:41 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1282 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:01:41.730381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:41.731096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:41.742221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.742791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.743433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.744234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.744752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.781930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-82a8de45-c8a5-4bcf-a1a7-947bbb47cdc4 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/315fd8d2-9660-4848-a58f-565b28e6e3ce" status: 200 len: 194 microversion: 2.1 time: 0.055789 Oct 09 01:01:41.782530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 349/694] 10.222.0.33 () {58 vars in 1413 bytes} [Sun Oct 9 01:01:41 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/315fd8d2-9660-4848-a58f-565b28e6e3ce => generated 194 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:41.796152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:41.796643 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:41.805320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.805803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.806434 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.807178 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:41.807892 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:41.808411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:41.923112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lazy-loading 'services' on Instance uuid bf1a22b3-a821-421e-8d9e-93190101a78d {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:41.989071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-1cf993a8-dd67-432d-84ca-320260611eeb req-a15ca2ea-c075-4cb3-9c88-39ba254589ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "edbfc046-964c-46ac-9d30-5198c846e9b5", "name": "network-vif-plugged", "status": "completed", "tag": "92161e7f-b2e3-4be5-9ff8-073cd2234c8e"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:42.018587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1cf993a8-dd67-432d-84ca-320260611eeb req-a15ca2ea-c075-4cb3-9c88-39ba254589ef service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:42.019550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1cf993a8-dd67-432d-84ca-320260611eeb req-a15ca2ea-c075-4cb3-9c88-39ba254589ef service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:42.021281 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1cf993a8-dd67-432d-84ca-320260611eeb req-a15ca2ea-c075-4cb3-9c88-39ba254589ef service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:42.044439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4a2a2d10-19d3-4886-84c0-43a8ff2cfcd3 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/315fd8d2-9660-4848-a58f-565b28e6e3ce" status: 202 len: 0 microversion: 2.1 time: 0.254103 Oct 09 01:01:42.046183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 346/695] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:01:41 2022] DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments/315fd8d2-9660-4848-a58f-565b28e6e3ce => generated 0 bytes in 257 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:01:42.049847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-1cf993a8-dd67-432d-84ca-320260611eeb req-a15ca2ea-c075-4cb3-9c88-39ba254589ef service nova] Creating event network-vif-plugged:92161e7f-b2e3-4be5-9ff8-073cd2234c8e for instance edbfc046-964c-46ac-9d30-5198c846e9b5 on hv-cinder-maste Oct 09 01:01:42.059433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-1cf993a8-dd67-432d-84ca-320260611eeb req-a15ca2ea-c075-4cb3-9c88-39ba254589ef service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076665 Oct 09 01:01:42.060432 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 350/696] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:42.540700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:42.541498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:42.547811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:42.548298 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:42.548735 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:42.549429 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:42.549884 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:42.550310 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:42.580819 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:42.581672 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:42.588044 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:42.588525 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:42.588960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:42.589815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:42.590082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:42.590649 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:42.700238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:42.738275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:42.741350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:42.742780 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:42.746846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6b92ac8a-4e95-4aa3-8bc0-72927b56cb9e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1334 microversion: 2.20 time: 0.170506 Oct 09 01:01:42.747993 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 351/697] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:42 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1334 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:01:42.761350 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-62646ecb-c9e4-4515-8947-4d27a9e628ed tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.224756 Oct 09 01:01:42.762115 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 347/698] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:42 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:43.774466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:43.775320 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:43.782830 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:43.783315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:43.783790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:43.784483 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:43.784936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:43.785307 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:43.785736 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:43.785907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:43.792251 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:43.792726 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:43.793270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:43.793932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:43.794386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:43.794850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:43.850009 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:43.859161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:43.860990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:43.862418 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7fa8d508-76db-4b5a-bc53-ff90de832789 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1334 microversion: 2.20 time: 0.093858 Oct 09 01:01:43.862984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 352/699] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:43 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1334 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:43.940604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:43.953823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-24e1b347-1f13-4ea5-9c38-257c27fa1292 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1265 microversion: 2.1 time: 0.172111 Oct 09 01:01:43.954495 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 348/700] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:43 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1265 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:44.240364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.241056 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:44.247154 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.247752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.248252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.249017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.249545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.250210 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.273748 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9b538303-8389-4ed6-90c9-dd4f8afc04d4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/ips" status: 200 len: 150 microversion: 2.1 time: 0.136202 Oct 09 01:01:44.274388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 353/701] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 01:01:44 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/ips => generated 150 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:44.307961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.310372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:44.316980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.317471 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.317914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.318608 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.319116 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.319543 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.349863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-eb1f57d4-bc6b-4262-ba26-d63fc6f204e7 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045778 Oct 09 01:01:44.350341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 349/702] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:44 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:01:44.474321 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.477086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:44.484525 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.485279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.485944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.487069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.487767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.488504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.530336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-eca0ee00-317d-46bd-9b46-d493c5237fdd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.060506 Oct 09 01:01:44.531657 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 354/703] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:44 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d/os-volume_attachments => generated 25 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:01:44.633400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.633710 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:44.640062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.640484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.640937 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.641639 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.642096 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.642516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.675400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:01:44.762388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7692f8f1-20cd-49ce-99b5-3efcb33c6459 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 204 len: 0 microversion: 2.1 time: 0.132822 Oct 09 01:01:44.762962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 350/704] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:44 2022] DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:01:44.771756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.772358 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:44.778524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.779099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.779410 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.780106 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.780553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.780968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.888686 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.889459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:44.897192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.897794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.898111 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.898811 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:44.899445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:44.899987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:44.970660 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:44.980592 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:44.982265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:44.983693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8551a748-b8b4-4d6d-aef2-cdc22bef2e98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1340 microversion: 2.20 time: 0.099279 Oct 09 01:01:44.984966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 351/705] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:44 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1340 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:44.998458 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:44.999083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:45.001457 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:45.007528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:45.008083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:45.008524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:45.009237 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:45.009854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:45.012137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:45.015119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-26e3d28a-66cf-4177-bf15-111c5521de60 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1674 microversion: 2.1 time: 0.247513 Oct 09 01:01:45.016293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 355/706] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:44 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1674 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:45.391886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:45.400497 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-27313dcf-db42-44fb-a864-0d53f2cefc25 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1452 microversion: 2.1 time: 0.414228 Oct 09 01:01:45.401331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 352/707] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:44 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1452 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:45.428585 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:45.429234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:45.436246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:45.436775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:45.437458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:45.438300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:45.438875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:45.439477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:45.658631 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:45.667326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fb2b0624-4ee5-421e-9aea-fee10d675ab5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1452 microversion: 2.1 time: 0.243531 Oct 09 01:01:45.671617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 356/708] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:45 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1452 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:45.675559 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3bf011bf-7bb1-4f6b-8c07-998a10c0a2cc"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:45.675559 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:45.680920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:45.681405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:45.681840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:45.682553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:45.683006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:45.683428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:46.011238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:46.011808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:46.018422 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:46.018912 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:46.019347 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:46.020081 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:46.020533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:46.020958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:46.299220 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:46.302356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:46.309783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:46.311230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:46.312517 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9b8a75da-bb22-4fe6-8834-dd5ba8f6a788 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.305360 Oct 09 01:01:46.313433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 357/709] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:46 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:01:46.318125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:46.318730 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:46.324545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:46.325038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:46.325469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:46.326156 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:46.326611 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:46.327031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:46.520126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:46.528089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f1103e3e-2505-4a82-9ede-d5d56a73bf09 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1674 microversion: 2.1 time: 0.213614 Oct 09 01:01:46.528589 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 358/710] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:46 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1674 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:46.886400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f25cde86-3ea4-4537-a422-5a02b10cb4cb tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "POST /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.217637 Oct 09 01:01:46.887003 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 353/711] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:01:45 2022] POST /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments => generated 194 bytes in 1219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:01:47.338699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:47.339352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:47.345799 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:47.346288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:47.346721 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:47.347410 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:47.348056 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:47.348484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:47.553664 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:47.554385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:47.561083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:47.561573 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:47.562010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:47.562717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:47.563177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:47.563617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:47.706664 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:47.713799 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=4,nova_cell1:UPDATE=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:47.716517 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e00ed5cd-d1a3-4e69-8d18-32f0a337446f tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1674 microversion: 2.1 time: 0.167691 Oct 09 01:01:47.717113 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 354/712] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:47 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1674 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:47.735270 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:47.743607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:47.745036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:47.746149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d7191362-bcad-4386-aa51-0430dea6c9ec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.411435 Oct 09 01:01:47.746709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 359/713] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:47 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:48.386559 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-f09596ef-9e32-4c32-8695-1fb8cdc093a5 req-889656b2-1827-48b1-b6dc-1ae2d21764c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:48.398865 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f09596ef-9e32-4c32-8695-1fb8cdc093a5 req-889656b2-1827-48b1-b6dc-1ae2d21764c0 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:48.399338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f09596ef-9e32-4c32-8695-1fb8cdc093a5 req-889656b2-1827-48b1-b6dc-1ae2d21764c0 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:48.400024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-f09596ef-9e32-4c32-8695-1fb8cdc093a5 req-889656b2-1827-48b1-b6dc-1ae2d21764c0 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:48.413489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-f09596ef-9e32-4c32-8695-1fb8cdc093a5 req-889656b2-1827-48b1-b6dc-1ae2d21764c0 service nova] Creating event network-changed:a48d8008-24c8-488e-9a77-3b3d2c3b71b4 for instance 98a78205-334d-469d-8eb4-26c151df1616 on hv-cinder-maste Oct 09 01:01:48.418275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-f09596ef-9e32-4c32-8695-1fb8cdc093a5 req-889656b2-1827-48b1-b6dc-1ae2d21764c0 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036340 Oct 09 01:01:48.419041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 355/714] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:01:48.740292 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:48.741012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:48.747548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:48.748272 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:48.748577 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:48.749314 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:48.749836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:48.750292 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:48.774056 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:48.775218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:48.785068 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:48.786083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:48.786519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:48.787242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:48.787877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:48.788298 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:48.936602 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:48.944763 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:48.946212 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:48.947323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5f935381-3d85-401b-ab6d-8d608a092e64 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.179481 Oct 09 01:01:48.956705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 356/715] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:48 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:01:49.144187 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:49.152857 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8e35f373-a2d5-49eb-9977-b6f12b3c34f8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1674 microversion: 2.1 time: 0.417012 Oct 09 01:01:49.153425 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 360/716] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:48 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1674 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:49.973342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:49.974499 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:49.981312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:49.982063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:49.982926 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:49.983823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:49.984473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:49.985084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:50.136709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:50.146460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:50.148012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:50.149131 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-74536fde-c3e5-4fc9-9f97-7c062f8766c3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.180388 Oct 09 01:01:50.149680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 357/717] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:49 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:50.177103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:50.177712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:50.180252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=4 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:50.186697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:50.187186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:50.187637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:50.188334 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:50.188957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:50.189385 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:50.266699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1512c91201f14b75a9be35b89c2097ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40e631c5e320489cac9cea85118c6263', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:50.277890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-58d041e7-73b0-40d9-9e20-7e3023e6ccbd tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 200 len: 1323 microversion: 2.1 time: 0.104540 Oct 09 01:01:50.278578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 361/718] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:50 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 1323 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:50.728890 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bf1a22b3-a821-421e-8d9e-93190101a78d", "name": "network-vif-deleted", "tag": "0b9a0bca-1f0d-4dc1-a233-b41357e21184"}, {"name": "network-changed", "server_uuid": "edbfc046-964c-46ac-9d30-5198c846e9b5", "tag": "92161e7f-b2e3-4be5-9ff8-073cd2234c8e"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:50.742236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:50.742723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:50.743147 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:50.759999 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] Creating event network-vif-deleted:0b9a0bca-1f0d-4dc1-a233-b41357e21184 for instance bf1a22b3-a821-421e-8d9e-93190101a78d on hv-cinder-maste Oct 09 01:01:50.763899 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] Creating event network-changed:92161e7f-b2e3-4be5-9ff8-073cd2234c8e for instance edbfc046-964c-46ac-9d30-5198c846e9b5 on hv-cinder-maste Oct 09 01:01:50.780105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-536824b2-0cd6-4fbe-90d3-cc76cc5da2d8 req-9f7f7773-c602-43e8-8af0-39168582352d service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.056892 Oct 09 01:01:50.781421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 358/719] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:50 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:51.174626 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:51.175327 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:51.182970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:51.183697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:51.187339 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:51.190304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:51.190856 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:51.191520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:51.301861 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:51.302653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:51.309331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:51.309821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:51.310262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:51.310963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:51.311422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:51.311878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:51.327813 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] HTTP exception thrown: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Request method failure captured: Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d HTTP/1.1 Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwalz5AigjQALstJl8FsJmuzD3g_ClB_VYUuUUBcr-err98-DsiSVUJM2XJMdcTEVZoUZCtrt9Dg5SkbdtyQ2v5kfPi91WnRH5l3IHEmhLcsHiTfQgJqyt2C8bqkh8eQaUJRBi2s49vd5c_FIxo3rJE63U7Ut34lA5pMGq0cWwewEI Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:01:51.329535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_40e631c5e320489cac9cea85118c6263"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/40e631c5e320489cac9cea85118c6263"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 1512c91201f14b75a9be35b89c2097ef Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:51.330891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'bf1a22b3-a821-421e-8d9e-93190101a78d'} Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:51.331900 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:51.334150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:51.335996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Returning 404 to user: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:51.335996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-57e0ca1d-70b5-44d1-aab1-7b65fb67e771 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 404 len: 111 microversion: 2.1 time: 0.033545 Oct 09 01:01:51.335996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 359/720] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:51 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 01:01:51.339948 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b2ffca3b-4a03-4a45-84e4-2354c4a61d0e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '.version_select at 0x7f7268ab6b80>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:51.358317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b2ffca3b-4a03-4a45-84e4-2354c4a61d0e tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1646329788" status: 202 len: 0 microversion: 2.1 time: 0.021819 Oct 09 01:01:51.359117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 360/721] 10.222.0.33 () {58 vars in 1221 bytes} [Sun Oct 9 01:01:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1646329788 => generated 0 bytes in 23 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 01:01:51.518823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:51.546810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:51.548330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:51.549348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-871cdc8d-6251-4758-9dc3-4d81cb2a14b3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.378601 Oct 09 01:01:51.554326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 362/722] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:51 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:52.571348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:52.572094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=8 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:01:52.576466 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:52.577067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:52.585209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:52.585700 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:52.586143 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:52.586834 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:52.587293 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:52.587755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:52.716011 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:52.725079 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:52.726609 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:52.727761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5421f914-24e6-40f9-9a73-5c7c2733b3bb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.157350 Oct 09 01:01:52.728336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 361/723] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:52 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:53.752476 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:53.753080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:53.759454 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:53.759969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:53.760408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:53.761094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:53.761548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:53.762421 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:53.895890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:53.904422 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:53.905851 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:53.906979 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3f039141-8d59-4cab-98ed-34c379661543 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.158405 Oct 09 01:01:53.907532 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 363/724] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:53 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:54.059457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-4699f0f5-4cc2-4508-b0d6-76fae91bf976 req-06d18327-d07b-4201-9487-b98d3b5f0d1f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "name": "network-vif-plugged", "status": "completed", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:54.070223 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-4699f0f5-4cc2-4508-b0d6-76fae91bf976 req-06d18327-d07b-4201-9487-b98d3b5f0d1f service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.070708 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-4699f0f5-4cc2-4508-b0d6-76fae91bf976 req-06d18327-d07b-4201-9487-b98d3b5f0d1f service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.071141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-4699f0f5-4cc2-4508-b0d6-76fae91bf976 req-06d18327-d07b-4201-9487-b98d3b5f0d1f service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.083768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-4699f0f5-4cc2-4508-b0d6-76fae91bf976 req-06d18327-d07b-4201-9487-b98d3b5f0d1f service nova] Creating event network-vif-plugged:a48d8008-24c8-488e-9a77-3b3d2c3b71b4 for instance 98a78205-334d-469d-8eb4-26c151df1616 on hv-cinder-maste Oct 09 01:01:54.088729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-4699f0f5-4cc2-4508-b0d6-76fae91bf976 req-06d18327-d07b-4201-9487-b98d3b5f0d1f service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093416 Oct 09 01:01:54.089621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 362/725] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:54.289053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:54.289544 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:54.295678 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.296170 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.296608 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.297302 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.297758 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.298187 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.312266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] HTTP exception thrown: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Request method failure captured: Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d HTTP/1.1 Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwalz5AigjQALstJl8FsJmuzD3g_ClB_VYUuUUBcr-err98-DsiSVUJM2XJMdcTEVZoUZCtrt9Dg5SkbdtyQ2v5kfPi91WnRH5l3IHEmhLcsHiTfQgJqyt2C8bqkh8eQaUJRBi2s49vd5c_FIxo3rJE63U7Ut34lA5pMGq0cWwewEI Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:01:54.313960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_40e631c5e320489cac9cea85118c6263"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/40e631c5e320489cac9cea85118c6263"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 1512c91201f14b75a9be35b89c2097ef Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.315546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'bf1a22b3-a821-421e-8d9e-93190101a78d'} Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:54.316767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.318061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.320143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Returning 404 to user: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:54.320143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1621d8d1-9ce5-4747-aa8d-0ff10b3c9e94 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 404 len: 111 microversion: 2.1 time: 0.030901 Oct 09 01:01:54.320143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 364/726] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:54 2022] DELETE /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:54.324351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:54.325045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: bf1a22b3-a821-421e-8d9e-93190101a78d] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:54.331046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.331532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.332528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.333262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.333748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.334175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.347537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] HTTP exception thrown: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Request method failure captured: Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d HTTP/1.1 Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwalz5AigjQALstJl8FsJmuzD3g_ClB_VYUuUUBcr-err98-DsiSVUJM2XJMdcTEVZoUZCtrt9Dg5SkbdtyQ2v5kfPi91WnRH5l3IHEmhLcsHiTfQgJqyt2C8bqkh8eQaUJRBi2s49vd5c_FIxo3rJE63U7Ut34lA5pMGq0cWwewEI Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:01:54.349724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_40e631c5e320489cac9cea85118c6263"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/40e631c5e320489cac9cea85118c6263"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 1512c91201f14b75a9be35b89c2097ef Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'bf1a22b3-a821-421e-8d9e-93190101a78d'} Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:54.352005 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. Oct 09 01:01:54.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:54.354589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Returning 404 to user: Instance bf1a22b3-a821-421e-8d9e-93190101a78d could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:54.354589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e23296f9-0ac9-4c0d-8286-c68f553b9c33 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d" status: 404 len: 111 microversion: 2.1 time: 0.030880 Oct 09 01:01:54.354589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 363/727] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:54 2022] GET /compute/v2.1/servers/bf1a22b3-a821-421e-8d9e-93190101a78d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:54.581037 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:54.581518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:54.587928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.588401 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.588828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.589500 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.589943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.590360 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.670008 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'services' on Instance uuid e0d9a7c4-b33c-42c1-9042-dfd357bc3932 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:01:54.784071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-11e6da83-cb89-4be9-8551-9928cc2f6ffd tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments/3bf011bf-7bb1-4f6b-8c07-998a10c0a2cc" status: 202 len: 0 microversion: 2.1 time: 0.206666 Oct 09 01:01:54.784666 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 365/728] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:01:54 2022] DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments/3bf011bf-7bb1-4f6b-8c07-998a10c0a2cc => generated 0 bytes in 208 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:01:54.793105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:54.795250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:54.801246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.801729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.802172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.802854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.803309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.803764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.833702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-da553fc6-2d8a-443a-98b1-0de630c3c00d tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044571 Oct 09 01:01:54.834253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 364/729] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:54 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:54.933210 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:54.933902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:54.940027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.940507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.940939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:54.942243 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:54.942694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:54.943117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.076828 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:55.084550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:55.085988 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:55.087093 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-41fe845d-7b73-4ae2-ae06-be8315a6dda5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1407 microversion: 2.20 time: 0.158168 Oct 09 01:01:55.087681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 366/730] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:54 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1407 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:01:55.496549 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:55.497023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:55.503278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:55.503783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:55.504217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.505031 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:55.505478 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:55.505897 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.518990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] HTTP exception thrown: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Request method failure captured: Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 HTTP/1.1 Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwalz5AigjQALstJl8FsJmuzD3g_ClB_VYUuUUBcr-err98-DsiSVUJM2XJMdcTEVZoUZCtrt9Dg5SkbdtyQ2v5kfPi91WnRH5l3IHEmhLcsHiTfQgJqyt2C8bqkh8eQaUJRBi2s49vd5c_FIxo3rJE63U7Ut34lA5pMGq0cWwewEI Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:01:55.520760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_40e631c5e320489cac9cea85118c6263"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/40e631c5e320489cac9cea85118c6263"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 1512c91201f14b75a9be35b89c2097ef Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.522305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '3247d2ef-8e30-4cee-b097-1957a0a3de81'} Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:55.524041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.525638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.527388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Returning 404 to user: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:55.527388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-77fb6923-2608-46e8-a143-021440b46a59 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 404 len: 111 microversion: 2.1 time: 0.029433 Oct 09 01:01:55.527388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 365/731] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:55 2022] DELETE /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:55.530787 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:55.531383 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] [instance: 3247d2ef-8e30-4cee-b097-1957a0a3de81] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:55.537834 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:55.538309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:55.538914 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.539786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:55.540387 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:55.540959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.557371 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] HTTP exception thrown: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Request method failure captured: Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 HTTP/1.1 Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwalz5AigjQALstJl8FsJmuzD3g_ClB_VYUuUUBcr-err98-DsiSVUJM2XJMdcTEVZoUZCtrt9Dg5SkbdtyQ2v5kfPi91WnRH5l3IHEmhLcsHiTfQgJqyt2C8bqkh8eQaUJRBi2s49vd5c_FIxo3rJE63U7Ut34lA5pMGq0cWwewEI Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:01:55.559099 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_40e631c5e320489cac9cea85118c6263"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/40e631c5e320489cac9cea85118c6263"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/40e631c5e320489cac9cea85118c6263"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 40e631c5e320489cac9cea85118c6263 Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1644302101 Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 1512c91201f14b75a9be35b89c2097ef Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeTestJSON-1644302101-project Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.560707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '3247d2ef-8e30-4cee-b097-1957a0a3de81'} Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:01:55.561594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. Oct 09 01:01:55.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:01:55.564506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] Returning 404 to user: Instance 3247d2ef-8e30-4cee-b097-1957a0a3de81 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:01:55.564605 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-20cd14dd-5923-4848-ba11-3a12776865d8 tempest-AttachVolumeTestJSON-1644302101 tempest-AttachVolumeTestJSON-1644302101-project] 10.222.0.33 "GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81" status: 404 len: 111 microversion: 2.1 time: 0.035615 Oct 09 01:01:55.568693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 367/732] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:55 2022] GET /compute/v2.1/servers/3247d2ef-8e30-4cee-b097-1957a0a3de81 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:01:55.846298 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:55.849270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:55.855081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:55.855558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:55.856022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.856706 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:55.857157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:55.857585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:55.888230 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bc91a50a-ce4a-4f10-afd6-806f48fc5cd6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045859 Oct 09 01:01:55.888785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 366/733] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:55 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:01:56.111971 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:56.112580 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:56.118867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:56.119350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:56.119832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:56.120529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:56.120980 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:56.121407 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:56.255314 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:56.263388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:56.264951 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:56.266192 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-55c4a880-0206-42d1-a774-ff540a0c82cb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1600 microversion: 2.20 time: 0.158247 Oct 09 01:01:56.266742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 368/734] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:01:56 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1600 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:01:56.291636 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:56.292162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:56.298002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:56.298485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:56.298923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:56.299613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:56.300074 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:56.300496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:56.413094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d478c29f-4e72-4c87-8253-628df65dc96e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.125368 Oct 09 01:01:56.413682 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 367/735] 10.222.0.33 () {60 vars in 1316 bytes} [Sun Oct 9 01:01:56 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-interface => generated 285 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:01:56.901767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:56.904840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:56.911345 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:56.911887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:56.912518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:56.913525 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:56.914373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:56.914801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:56.947006 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4f39ab9f-830d-4242-b73c-8ac874df19af tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049451 Oct 09 01:01:56.947560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 369/736] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:01:56 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:01:57.014365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:57.014863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:57.021692 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:57.022214 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:57.022655 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:57.023348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:57.023893 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:57.024348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:57.050121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:01:57.125081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5acd99e8-db43-4687-9957-8c1645417409 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 204 len: 0 microversion: 2.1 time: 0.114707 Oct 09 01:01:57.125885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 368/737] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:01:57 2022] DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:01:57.134492 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:57.135094 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:57.141844 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:57.142330 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:57.142912 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:57.143636 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:57.144098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:57.144547 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:57.280086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:57.288324 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8275af8b-75d2-428f-a266-b7ddc8bcf289 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1680 microversion: 2.1 time: 0.157684 Oct 09 01:01:57.288904 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 370/738] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:57 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:01:58.313445 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:58.323453 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:58.324702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-1b08c229-e497-4764-9400-3d72f890117a req-a400651f-5cc1-48b8-9871-9043acc97006 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:01:58.342081 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:58.345548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:58.346257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:58.348943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:58.350276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:58.353163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:58.363509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-1b08c229-e497-4764-9400-3d72f890117a req-a400651f-5cc1-48b8-9871-9043acc97006 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:58.364987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-1b08c229-e497-4764-9400-3d72f890117a req-a400651f-5cc1-48b8-9871-9043acc97006 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:58.365898 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-1b08c229-e497-4764-9400-3d72f890117a req-a400651f-5cc1-48b8-9871-9043acc97006 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:58.385405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-1b08c229-e497-4764-9400-3d72f890117a req-a400651f-5cc1-48b8-9871-9043acc97006 service nova] Creating event network-changed:a48d8008-24c8-488e-9a77-3b3d2c3b71b4 for instance 98a78205-334d-469d-8eb4-26c151df1616 on hv-cinder-maste Oct 09 01:01:58.404568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-1b08c229-e497-4764-9400-3d72f890117a req-a400651f-5cc1-48b8-9871-9043acc97006 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.175551 Oct 09 01:01:58.408187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 369/739] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:01:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:01:58.551757 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:58.561436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a7fe1b25-3e75-4fee-918f-1698f72629e5 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1680 microversion: 2.1 time: 0.252356 Oct 09 01:01:58.562326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 371/740] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:58 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1680 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:01:59.587912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:01:59.589219 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:01:59.595679 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:59.596376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:59.596998 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:59.598478 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:01:59.599084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:01:59.599680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:01:59.863905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:01:59.874962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d44a7434-207d-4a8c-bb05-2ca74aa178e9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1680 microversion: 2.1 time: 0.291707 Oct 09 01:01:59.875535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 370/741] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:01:59 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1680 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:02:00.408536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-68e9fc7e-af87-4ffd-974c-928f306d65a0 req-5ec18f02-786f-4b26-a701-0f8739997033 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e0d9a7c4-b33c-42c1-9042-dfd357bc3932", "name": "network-vif-deleted", "tag": "bf1ecde5-a035-439a-96b4-f7a4880b70e4"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:00.415359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:02:00.421943 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-68e9fc7e-af87-4ffd-974c-928f306d65a0 req-5ec18f02-786f-4b26-a701-0f8739997033 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:00.422442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-68e9fc7e-af87-4ffd-974c-928f306d65a0 req-5ec18f02-786f-4b26-a701-0f8739997033 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:00.422881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-68e9fc7e-af87-4ffd-974c-928f306d65a0 req-5ec18f02-786f-4b26-a701-0f8739997033 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:00.440694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-68e9fc7e-af87-4ffd-974c-928f306d65a0 req-5ec18f02-786f-4b26-a701-0f8739997033 service nova] Creating event network-vif-deleted:bf1ecde5-a035-439a-96b4-f7a4880b70e4 for instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 on hv-cinder-maste Oct 09 01:02:00.450574 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-68e9fc7e-af87-4ffd-974c-928f306d65a0 req-5ec18f02-786f-4b26-a701-0f8739997033 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046188 Oct 09 01:02:00.453262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 372/742] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:02:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:02:00.908702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:00.910807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:00.924501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:00.925225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:00.926312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:00.927237 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:00.928675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:00.929325 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:00.965363 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:00.965930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:00.966562 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:00.967331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lazy-loading 'fault' on Instance uuid e0d9a7c4-b33c-42c1-9042-dfd357bc3932 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:02:01.029818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b027fb8adadc488e975c4541e7ee6626', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7d8e82f58814da6b8c08bfa04955d4c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:01.040197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-99be8ae0-9706-42ef-b255-2677ffd945d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 200 len: 1330 microversion: 2.1 time: 0.142407 Oct 09 01:02:01.055952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 371/743] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:02:00 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 1330 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:02:02.064827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:02.066642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:02.092238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:02.092726 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:02.093319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:02.094379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:02.095519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:02.096129 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:02.111674 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 HTTP/1.1 Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:02.113413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:02.115017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'e0d9a7c4-b33c-42c1-9042-dfd357bc3932'} Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:02.115852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:02.117080 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:02.118221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:02.118221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-778d7772-7f6e-4b42-9e4d-eb7a181727d1 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 404 len: 111 microversion: 2.1 time: 0.054636 Oct 09 01:02:02.118221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 373/744] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:02:02 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:02.124984 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-abf2a9e6-e323-451e-a961-02980377dc5f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '.version_select at 0x7f7268b45820>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:02.147463 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-abf2a9e6-e323-451e-a961-02980377dc5f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-33835709" status: 202 len: 0 microversion: 2.1 time: 0.027546 Oct 09 01:02:02.148070 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 372/745] 10.222.0.33 () {58 vars in 1215 bytes} [Sun Oct 9 01:02:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-33835709 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 01:02:04.170214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:04.170703 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:04.177539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:04.178022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:04.178460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:04.179148 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:04.179623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:04.180051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:04.194212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 HTTP/1.1 Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:04.195785 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.197388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'e0d9a7c4-b33c-42c1-9042-dfd357bc3932'} Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:04.198265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.201567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:04.201567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4889567d-3352-4eed-95a9-cb04f4b14aa9 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 404 len: 111 microversion: 2.1 time: 0.030793 Oct 09 01:02:04.201567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 374/746] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:02:04 2022] DELETE /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:04.206877 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:04.207605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: e0d9a7c4-b33c-42c1-9042-dfd357bc3932] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:04.214973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:04.215454 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:04.215906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:04.216595 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:04.217044 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:04.217464 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:04.231118 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 HTTP/1.1 Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:04.232816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.234396 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'e0d9a7c4-b33c-42c1-9042-dfd357bc3932'} Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:04.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. Oct 09 01:02:04.236385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:04.237571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance e0d9a7c4-b33c-42c1-9042-dfd357bc3932 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:04.237571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e77eab80-e9a8-4901-847e-5fc85a97b693 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932" status: 404 len: 111 microversion: 2.1 time: 0.032375 Oct 09 01:02:04.237571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 373/747] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:02:04 2022] GET /compute/v2.1/servers/e0d9a7c4-b33c-42c1-9042-dfd357bc3932 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:05.388845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:05.389348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:05.399617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.399617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.399617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.399617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.399999 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.399999 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.415421 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 HTTP/1.1 Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:05.416961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.418520 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '188a0b8d-23e2-4769-94bf-85277c3b0b66'} Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:05.419664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.421102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.422493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:05.422493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c3dd4fd4-05ec-4478-a9b3-19662487210e tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 404 len: 111 microversion: 2.1 time: 0.034122 Oct 09 01:02:05.422493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 375/748] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:02:05 2022] DELETE /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:05.430714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:05.431316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 188a0b8d-23e2-4769-94bf-85277c3b0b66] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:05.437613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.438091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.438525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.439217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.439690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.440107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.453497 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 HTTP/1.1 Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:05.454997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '188a0b8d-23e2-4769-94bf-85277c3b0b66'} Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:05.458012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. Oct 09 01:02:05.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.461001 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance 188a0b8d-23e2-4769-94bf-85277c3b0b66 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:05.461001 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c5635216-741a-47a6-ad25-7e2c7e75607f tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66" status: 404 len: 111 microversion: 2.1 time: 0.029509 Oct 09 01:02:05.461001 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 374/749] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:02:05 2022] GET /compute/v2.1/servers/188a0b8d-23e2-4769-94bf-85277c3b0b66 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:05.465112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:05.465647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:05.471342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.471847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.472905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.473581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.474026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.474441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.488535 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a HTTP/1.1 Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:05.490035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.491660 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '9988ceee-fd6b-43f1-ae12-74b98a9f128a'} Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:05.492560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.494188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.495996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:05.495996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fcd3b3e3-997a-4745-b011-5f4a14811cf4 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 404 len: 111 microversion: 2.1 time: 0.030040 Oct 09 01:02:05.495996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 376/750] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:02:05 2022] DELETE /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:05.501140 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:05.501750 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: 9988ceee-fd6b-43f1-ae12-74b98a9f128a] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:05.508021 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.508671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.508971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.509671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:05.510217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:05.510728 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:05.526572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a HTTP/1.1 Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:05.528117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.529695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '9988ceee-fd6b-43f1-ae12-74b98a9f128a'} Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:05.530474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. Oct 09 01:02:05.531743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:05.532796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance 9988ceee-fd6b-43f1-ae12-74b98a9f128a could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:05.532796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-01312fe2-80f9-44ff-8fe9-81f24ccf9a6c tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a" status: 404 len: 111 microversion: 2.1 time: 0.033299 Oct 09 01:02:05.532796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 375/751] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:02:05 2022] GET /compute/v2.1/servers/9988ceee-fd6b-43f1-ae12-74b98a9f128a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:06.679399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:06.679936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:06.686492 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:06.686966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:06.687631 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:06.688309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:06.688753 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:06.689171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:06.702377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e HTTP/1.1 Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:06.703915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.705463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'acc5e0e3-a218-4f87-b4cc-2efeb32e281e'} Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:06.706377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.707933 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.709406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:06.709406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8e8be3dc-f19e-4f11-b24b-117dcc50c205 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 404 len: 111 microversion: 2.1 time: 0.030221 Oct 09 01:02:06.709406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 377/752] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:02:06 2022] DELETE /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:06.713874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:06.714485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] [instance: acc5e0e3-a218-4f87-b4cc-2efeb32e281e] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:06.721014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:06.721495 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:06.721976 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:06.723256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:06.723813 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:06.724245 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:06.737570 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] HTTP exception thrown: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Request method failure captured: Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e HTTP/1.1 Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhwmu47WTab0B2Ey9yUvE9EOJV-99dMm7tmcRWJ-XNy1MexJIwftChtpw9kBz2FEmYohNoeU8c0VJ82XmoXX9vzwP2vJ2_CFNcUClMDlKMvQsB2UmEdCKdapQhdqJe_Wd9kGHF-tqcjxnlYfG4y44qZ6YqrIT7Z-j70Q__vyELvlVtM Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:06.739779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/a7d8e82f58814da6b8c08bfa04955d4c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: a7d8e82f58814da6b8c08bfa04955d4c Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1432185703 Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: b027fb8adadc488e975c4541e7ee6626 Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeNegativeTest-1432185703-project Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.741575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'acc5e0e3-a218-4f87-b4cc-2efeb32e281e'} Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:06.742502 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. Oct 09 01:02:06.744050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:06.745340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] Returning 404 to user: Instance acc5e0e3-a218-4f87-b4cc-2efeb32e281e could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:06.745340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d26954f0-d5cb-4c89-8835-052f70f019d6 tempest-AttachVolumeNegativeTest-1432185703 tempest-AttachVolumeNegativeTest-1432185703-project] 10.222.0.33 "GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e" status: 404 len: 111 microversion: 2.1 time: 0.031287 Oct 09 01:02:06.745340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 376/753] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:02:06 2022] GET /compute/v2.1/servers/acc5e0e3-a218-4f87-b4cc-2efeb32e281e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:11.446057 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0f3d0df4-69dc-493d-badc-5bd184c7efb0 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Action: 'create', calling method: .version_select at 0x7f7268a48820>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1219389169", "metadata": {"Type": "work"}}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:11.738190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0f3d0df4-69dc-493d-badc-5bd184c7efb0 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.409040 Oct 09 01:02:11.751369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1f94cb7b-70c1-4ebd-8f63-6b576a036cb0 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Calling method '.version_select at 0x7f7268b98af0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:11.752026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 378/754] 10.222.0.33 () {60 vars in 1156 bytes} [Sun Oct 9 01:02:11 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 424 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 09 01:02:11.810364 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1f94cb7b-70c1-4ebd-8f63-6b576a036cb0 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb" status: 200 len: 353 microversion: 2.1 time: 0.062792 Oct 09 01:02:11.810913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 377/755] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 01:02:11 2022] GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb => generated 353 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:02:12.825686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-19d630db-e6ba-4a84-9c3d-fc2ee556b687 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Calling method '.version_select at 0x7f7268a95af0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:12.873336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-19d630db-e6ba-4a84-9c3d-fc2ee556b687 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb" status: 200 len: 354 microversion: 2.1 time: 0.051900 Oct 09 01:02:12.873931 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 379/756] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 01:02:12 2022] GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:02:12.887470 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-15fb8f9d-7998-4ade-b0f2-5cff6c97d062 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Calling method '.version_select at 0x7f72689830d0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:12.934783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-15fb8f9d-7998-4ade-b0f2-5cff6c97d062 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb" status: 200 len: 354 microversion: 2.1 time: 0.051754 Oct 09 01:02:12.935508 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 378/757] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 01:02:12 2022] GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:02:12.948450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-027b7b56-7b83-4a1e-b146-eec9184a4662 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Calling method '.version_select at 0x7f72689ca9d0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:13.009100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-027b7b56-7b83-4a1e-b146-eec9184a4662 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb" status: 202 len: 0 microversion: 2.1 time: 0.064306 Oct 09 01:02:13.009672 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 380/758] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 01:02:12 2022] DELETE /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb => generated 0 bytes in 66 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:02:13.017589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c9c7f349-4795-4ee0-87b1-415af9c406b3 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Calling method '.version_select at 0x7f72689855e0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:13.065702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c9c7f349-4795-4ee0-87b1-415af9c406b3 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb" status: 200 len: 353 microversion: 2.1 time: 0.051735 Oct 09 01:02:13.066245 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 379/759] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 01:02:13 2022] GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb => generated 353 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:02:14.080563 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cd5f0f52-f2ec-4bd1-a9cd-a42e5be890b1 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Calling method '.version_select at 0x7f72689e6b80>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:14.108342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-cd5f0f52-f2ec-4bd1-a9cd-a42e5be890b1 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] HTTP exception thrown: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cd5f0f52-f2ec-4bd1-a9cd-a42e5be890b1 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Request method failure captured: Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb HTTP/1.1 Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh2T1fkGY7mN5N0tMkm6XUJuEPDQdU1ZnZ3bu7TrBpJ0k1aA4SZhTQOfeI-jtJi0H0fpq9ncRLnBHNPkP1HzV5aIgro5RyWlnYbcL7VfVYSdgUTxYgTtjcCy32zieLmGlEef1yp8eHmXr-F4aIvlu3y4hUnceAM6n67YcYDXJYEIjLU Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 97b9086fac714440938c6c50bd5026c3 Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-VolumesGetTestJSON-372869259 Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:14.111545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_97b9086fac714440938c6c50bd5026c3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/97b9086fac714440938c6c50bd5026c3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/97b9086fac714440938c6c50bd5026c3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/97b9086fac714440938c6c50bd5026c3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-VolumesGetTestJSON-372869259 Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 97b9086fac714440938c6c50bd5026c3 Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-VolumesGetTestJSON-372869259 Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-VolumesGetTestJSON-372869259-project Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 05e4d867a47c4c6bbfeeb98041cc1dc1 Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-VolumesGetTestJSON-372869259-project Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '9f2ff9f9-2399-4740-9ec0-e434fd04a0bb'} Oct 09 01:02:14.113664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. (HTTP 404) (Request-ID: req-67a1c510-d942-4f88-ab5d-33f12d6f1488) Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:14.114586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 01:02:14.116035 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 09 01:02:14.117136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cd5f0f52-f2ec-4bd1-a9cd-a42e5be890b1 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] Returning 404 to user: Volume 9f2ff9f9-2399-4740-9ec0-e434fd04a0bb could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cd5f0f52-f2ec-4bd1-a9cd-a42e5be890b1 tempest-VolumesGetTestJSON-372869259 tempest-VolumesGetTestJSON-372869259-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb" status: 404 len: 109 microversion: 2.1 time: 0.037405 Oct 09 01:02:14.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 381/760] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 01:02:14 2022] GET /compute/v2.1/os-volumes/9f2ff9f9-2399-4740-9ec0-e434fd04a0bb => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:15.499194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4842d265-781d-4f48-b5fe-9983751c4e93 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Action: 'create', calling method: .version_select at 0x7f72689c8ca0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-44463666", "metadata": {"Type": "work"}}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4842d265-781d-4f48-b5fe-9983751c4e93 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Request method failure captured: Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 120 Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh2XFgXznHesYys9OY0rURv6QkpZDa1D-xel-qEeJQKXGtPZSztpcMDiwIyFr3d_y53zruqfCEI3A9QZVOePisRnVUnCXmoq1UImM44sEv5BBXXt5t_GpVgezxSBMyuQaxnoWEm0pdpoMwpl3Bzql7fXGT9lCUVUjKSxiu_YGuRVvNg Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:15.509579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5562380a08c440759380988a5b2d0b7c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 082a4dc285c942eca98c4c3ce78193f4 Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-44463666", "metadata": {"Type": "work"}}} Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 01:02:15.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-44463666', 'metadata': {'Type': 'work'}}}} Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise error Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi '#$%' Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:15.512032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 09 01:02:15.513371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 09 01:02:15.514535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 09 01:02:15.514535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.514535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4842d265-781d-4f48-b5fe-9983751c4e93 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:15.514535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4842d265-781d-4f48-b5fe-9983751c4e93 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.136184 Oct 09 01:02:15.514535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 380/761] 10.222.0.33 () {60 vars in 1156 bytes} [Sun Oct 9 01:02:15 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 137 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:02:15.522553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6f0c7ee4-8f3a-404b-8613-06cc68c0c566 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Action: 'create', calling method: .version_select at 0x7f72689e6b80>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1724727976", "metadata": {"Type": "work"}}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6f0c7ee4-8f3a-404b-8613-06cc68c0c566 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Request method failure captured: Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 120 Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh2XFgXznHesYys9OY0rURv6QkpZDa1D-xel-qEeJQKXGtPZSztpcMDiwIyFr3d_y53zruqfCEI3A9QZVOePisRnVUnCXmoq1UImM44sEv5BBXXt5t_GpVgezxSBMyuQaxnoWEm0pdpoMwpl3Bzql7fXGT9lCUVUjKSxiu_YGuRVvNg Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:15.527295 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5562380a08c440759380988a5b2d0b7c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 082a4dc285c942eca98c4c3ce78193f4 Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1724727976", "metadata": {"Type": "work"}}} Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: Oct 09 01:02:15.529039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1724727976', 'metadata': {'Type': 'work'}}}} Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise error Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi '0' Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:15.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 09 01:02:15.531506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 09 01:02:15.532642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 09 01:02:15.532642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.532642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6f0c7ee4-8f3a-404b-8613-06cc68c0c566 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:15.532642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6f0c7ee4-8f3a-404b-8613-06cc68c0c566 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.011056 Oct 09 01:02:15.532642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 382/762] 10.222.0.33 () {60 vars in 1156 bytes} [Sun Oct 9 01:02:15 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:02:15.539108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-684aa0ab-d51f-4ff9-999b-2c0b28f87eca tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Action: 'create', calling method: .version_select at 0x7f7268985820>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1850196600", "metadata": {"Type": "work"}}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-684aa0ab-d51f-4ff9-999b-2c0b28f87eca tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Request method failure captured: Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 119 Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh2XFgXznHesYys9OY0rURv6QkpZDa1D-xel-qEeJQKXGtPZSztpcMDiwIyFr3d_y53zruqfCEI3A9QZVOePisRnVUnCXmoq1UImM44sEv5BBXXt5t_GpVgezxSBMyuQaxnoWEm0pdpoMwpl3Bzql7fXGT9lCUVUjKSxiu_YGuRVvNg Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:15.547737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5562380a08c440759380988a5b2d0b7c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 082a4dc285c942eca98c4c3ce78193f4 Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1850196600", "metadata": {"Type": "work"}}} Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 01:02:15.549194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1850196600', 'metadata': {'Type': 'work'}}}} Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise error Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi '' Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:15.552017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 09 01:02:15.553333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 09 01:02:15.554467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 09 01:02:15.554467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.554467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-684aa0ab-d51f-4ff9-999b-2c0b28f87eca tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:15.554467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-684aa0ab-d51f-4ff9-999b-2c0b28f87eca tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.014513 Oct 09 01:02:15.554467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 381/763] 10.222.0.33 () {60 vars in 1156 bytes} [Sun Oct 9 01:02:15 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:02:15.561022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0dec1624-18f9-47db-9e41-3d87d6b44017 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Calling method '.version_select at 0x7f7268934430>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:15.588948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-0dec1624-18f9-47db-9e41-3d87d6b44017 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] HTTP exception thrown: Volume tempest-invalid-463569303 could not be found. Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0dec1624-18f9-47db-9e41-3d87d6b44017 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Request method failure captured: Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-463569303 HTTP/1.1 Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh2XFgXznHesYys9OY0rURv6QkpZDa1D-xel-qEeJQKXGtPZSztpcMDiwIyFr3d_y53zruqfCEI3A9QZVOePisRnVUnCXmoq1UImM44sEv5BBXXt5t_GpVgezxSBMyuQaxnoWEm0pdpoMwpl3Bzql7fXGT9lCUVUjKSxiu_YGuRVvNg Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:15.591072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5562380a08c440759380988a5b2d0b7c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 082a4dc285c942eca98c4c3ce78193f4 Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Volume tempest-invalid-463569303 could not be found. Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': 'tempest-invalid-463569303'} Oct 09 01:02:15.592663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-463569303 could not be found. (HTTP 404) (Request-ID: req-728d53f6-5282-43f7-a909-b236720d42ee) Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.593548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 09 01:02:15.594675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-463569303 could not be found. Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:15.595893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-463569303 could not be found. Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-463569303 could not be found. Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0dec1624-18f9-47db-9e41-3d87d6b44017 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Returning 404 to user: Volume tempest-invalid-463569303 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:15.597693 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0dec1624-18f9-47db-9e41-3d87d6b44017 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/tempest-invalid-463569303" status: 404 len: 98 microversion: 2.1 time: 0.035876 Oct 09 01:02:15.599848 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 383/764] 10.222.0.33 () {58 vars in 1215 bytes} [Sun Oct 9 01:02:15 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-463569303 => generated 98 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 09 01:02:15.604259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-efa67bbb-6d96-47b2-a7ef-98737c725d09 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003678 Oct 09 01:02:15.604753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 382/765] 10.222.0.33 () {58 vars in 1140 bytes} [Sun Oct 9 01:02:15 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 09 01:02:15.614555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8d67d1f7-8c61-4089-bcce-fcc2922a89ff tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005121 Oct 09 01:02:15.615186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 384/766] 10.222.0.33 () {58 vars in 1137 bytes} [Sun Oct 9 01:02:15 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 09 01:02:15.623800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e4a9f65e-1878-41dc-8abb-67173a13efef tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Calling method '.version_select at 0x7f72689850d0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:15.653556 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-e4a9f65e-1878-41dc-8abb-67173a13efef tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] HTTP exception thrown: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e4a9f65e-1878-41dc-8abb-67173a13efef tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Request method failure captured: Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: DELETE /compute/v2.1/os-volumes/be64c4f7-2ff5-4472-be7b-d513b1712b9b HTTP/1.1 Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh2XFgXznHesYys9OY0rURv6QkpZDa1D-xel-qEeJQKXGtPZSztpcMDiwIyFr3d_y53zruqfCEI3A9QZVOePisRnVUnCXmoq1UImM44sEv5BBXXt5t_GpVgezxSBMyuQaxnoWEm0pdpoMwpl3Bzql7fXGT9lCUVUjKSxiu_YGuRVvNg Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:02:15.655482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5562380a08c440759380988a5b2d0b7c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 082a4dc285c942eca98c4c3ce78193f4 Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'be64c4f7-2ff5-4472-be7b-d513b1712b9b'} Oct 09 01:02:15.656847 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. (HTTP 404) (Request-ID: req-47207df9-d577-4b19-884c-ea0edb639cf6) Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.657649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 09 01:02:15.660081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 09 01:02:15.661638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e4a9f65e-1878-41dc-8abb-67173a13efef tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Returning 404 to user: Volume be64c4f7-2ff5-4472-be7b-d513b1712b9b could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e4a9f65e-1878-41dc-8abb-67173a13efef tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "DELETE /compute/v2.1/os-volumes/be64c4f7-2ff5-4472-be7b-d513b1712b9b" status: 404 len: 109 microversion: 2.1 time: 0.036984 Oct 09 01:02:15.663271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 383/767] 10.222.0.33 () {58 vars in 1248 bytes} [Sun Oct 9 01:02:15 2022] DELETE /compute/v2.1/os-volumes/be64c4f7-2ff5-4472-be7b-d513b1712b9b => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:15.667881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d9ed6408-f462-4836-9ae0-62e8f0eb8dc9 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Calling method '.version_select at 0x7f72689ca9d0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:15.696176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-d9ed6408-f462-4836-9ae0-62e8f0eb8dc9 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] HTTP exception thrown: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d9ed6408-f462-4836-9ae0-62e8f0eb8dc9 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Request method failure captured: Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/os-volumes/4640629f-bc18-4f1d-b3ec-0a00cffa5209 HTTP/1.1 Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh2XFgXznHesYys9OY0rURv6QkpZDa1D-xel-qEeJQKXGtPZSztpcMDiwIyFr3d_y53zruqfCEI3A9QZVOePisRnVUnCXmoq1UImM44sEv5BBXXt5t_GpVgezxSBMyuQaxnoWEm0pdpoMwpl3Bzql7fXGT9lCUVUjKSxiu_YGuRVvNg Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:02:15.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_5562380a08c440759380988a5b2d0b7c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/5562380a08c440759380988a5b2d0b7c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 5562380a08c440759380988a5b2d0b7c Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-VolumesNegativeTest-324148503 Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 082a4dc285c942eca98c4c3ce78193f4 Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-VolumesNegativeTest-324148503-project Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '4640629f-bc18-4f1d-b3ec-0a00cffa5209'} Oct 09 01:02:15.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. (HTTP 404) (Request-ID: req-8954d42f-7ef5-43dd-89d4-1410d631e7b6) Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.700266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 09 01:02:15.701650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 09 01:02:15.703255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d9ed6408-f462-4836-9ae0-62e8f0eb8dc9 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] Returning 404 to user: Volume 4640629f-bc18-4f1d-b3ec-0a00cffa5209 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d9ed6408-f462-4836-9ae0-62e8f0eb8dc9 tempest-VolumesNegativeTest-324148503 tempest-VolumesNegativeTest-324148503-project] 10.222.0.33 "GET /compute/v2.1/os-volumes/4640629f-bc18-4f1d-b3ec-0a00cffa5209" status: 404 len: 109 microversion: 2.1 time: 0.035546 Oct 09 01:02:15.705291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 385/768] 10.222.0.33 () {58 vars in 1245 bytes} [Sun Oct 9 01:02:15 2022] GET /compute/v2.1/os-volumes/4640629f-bc18-4f1d-b3ec-0a00cffa5209 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:02:33.645562 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=2,nova_api:DELETE=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:02:33.646316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:02:33.650970 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:33.651544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:33.661639 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:33.662129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:33.662565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:33.663276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:33.663738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:33.664169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:33.688786 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-448e0cd3-459c-4c94-a0d1-f4931d1f8ab1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/ips" status: 200 len: 153 microversion: 2.20 time: 0.045103 Oct 09 01:02:33.689352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 384/769] 10.222.0.33 () {60 vars in 1289 bytes} [Sun Oct 9 01:02:33 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/ips => generated 153 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:02:35.884308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=11 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:02:35.885368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:02:35.890365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:35.891084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:35.901890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:35.902374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:35.902807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:35.903496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:35.903969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:35.904391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:35.928278 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lazy-loading 'flavor' on Instance uuid 98a78205-334d-469d-8eb4-26c151df1616 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:02:36.130681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5949653b-bade-49e4-80a3-aa51863f54b7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/action" status: 202 len: 0 microversion: 2.20 time: 0.248073 Oct 09 01:02:36.131203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 386/770] 10.222.0.33 () {62 vars in 1319 bytes} [Sun Oct 9 01:02:35 2022] POST /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/action => generated 0 bytes in 249 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:02:36.140600 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:36.141209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:36.152535 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:36.153034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:36.153456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:36.154150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:36.154599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:36.155026 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:36.300632 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:36.309020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:36.310444 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:36.311654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-63ff50ae-a75b-4f88-9366-23f6c8ef0ada tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.174862 Oct 09 01:02:36.320042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 385/771] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:36 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:37.338451 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:37.339178 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:37.346578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:37.347559 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:37.348039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:37.348727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:37.349179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:37.349602 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:37.496002 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:37.504821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:37.506258 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:37.507406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2c627361-e33c-41a8-80ce-040d28a8d41b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.173435 Oct 09 01:02:37.508065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 387/772] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:37 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:38.533154 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:38.533696 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:38.540382 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:38.540859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:38.541299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:38.541986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:38.542438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:38.542860 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:38.681395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:38.690552 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:38.696299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:38.697490 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-07b7a6d2-71cc-4c64-9d3c-293ec87921e3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.168245 Oct 09 01:02:38.698045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 386/773] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:38 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:39.722521 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:39.723182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:39.729503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:39.729972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:39.730399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:39.731420 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:39.731892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:39.732309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:39.861364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:39.870537 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:39.871982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:39.873160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-caa95092-57e6-4520-92b7-d5ca754f94cc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.154850 Oct 09 01:02:39.873711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 388/774] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:39 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:40.898315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:40.898910 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:40.905557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:40.906028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:40.906464 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:40.907140 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:40.907586 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:40.908032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:41.035200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:41.043576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:41.045023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:41.046154 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a68d2a81-a3d4-4847-acc6-89ac57d64f24 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.151721 Oct 09 01:02:41.046855 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 387/775] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:40 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:42.071767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:42.072384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:42.078767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:42.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:42.079917 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:42.080606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:42.081573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:42.081993 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:42.210523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:42.219265 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:42.220747 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:42.222439 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2ef5e266-b0c6-4cb0-9e75-346b5727589f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.153996 Oct 09 01:02:42.222986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 389/776] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:42 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:43.247146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:43.247792 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:43.254676 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:43.255187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:43.255713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:43.256538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:43.257010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:43.257417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:43.385953 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:43.394489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:43.395958 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:43.397102 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b2683d4f-0361-4140-b3fd-79d086032aa9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.153826 Oct 09 01:02:43.397658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 388/777] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:43 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:44.423447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:44.424086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:44.430441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:44.430908 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:44.431331 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:44.432042 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:44.432487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:44.432997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:44.560257 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:44.568144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:44.569594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:44.570778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f9d6209b-f061-4177-9e49-8d03bc93956d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.151871 Oct 09 01:02:44.571276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 390/778] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:44 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:45.597217 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:45.597941 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:45.605479 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:45.605931 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:45.606476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:45.607173 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:45.607758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:45.608150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:45.811399 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:45.819930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:45.821506 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:45.822689 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bb1d868f-5cab-4dfd-97e2-50b601783abd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.230116 Oct 09 01:02:45.823372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 389/779] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:45 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:46.849403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:46.850947 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:46.857363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:46.857839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:46.858268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:46.858949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:46.859403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:46.859858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:46.988771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:46.997072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:46.998486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:46.999671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c962fd0b-8e8f-4123-b738-8095d6d969ca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.154431 Oct 09 01:02:47.000221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 391/780] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:46 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:48.026297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:48.026902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:48.034246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:48.034743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:48.035177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:48.035886 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:48.036334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:48.036754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:48.172918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:48.181542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:48.183023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:48.184203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-83b4bbd5-a6df-41e8-a93b-dfacad44c154 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.161997 Oct 09 01:02:48.184767 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 390/781] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:48 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:49.211365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:49.212168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:49.218921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:49.219401 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:49.219899 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:49.220596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:49.221041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:49.221488 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:49.358127 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:49.366078 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:49.367518 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:49.368692 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-153ea3c3-e4d0-4dc9-9bff-c5f16cf49d2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.163032 Oct 09 01:02:49.369240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 392/782] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:49 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:50.394909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:50.395446 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:50.402131 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:50.402615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:50.403054 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:50.403779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:50.404238 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:50.404665 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:50.532209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:50.542081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:50.543663 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:50.545002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f50a30ae-5eb0-4926-a7d8-9250fce1a78f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.154553 Oct 09 01:02:50.545830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 391/783] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:50 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:51.573219 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:51.573750 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:51.580116 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:51.580592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:51.581023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:51.582361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:51.582808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:51.583228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:51.706143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:51.714649 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:51.716118 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:51.717256 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b3974fcd-2525-433d-9500-177a51116733 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.147973 Oct 09 01:02:51.717807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 393/784] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:51 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:52.742425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:52.743029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:52.749654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:52.750135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:52.750623 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:52.751336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:52.751818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:52.752299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:52.898744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:52.907053 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:52.908523 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:52.910260 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-175001c6-44a3-4561-b974-9b369fe1010d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1746 microversion: 2.20 time: 0.171278 Oct 09 01:02:52.910819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 392/785] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:52 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1746 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:53.935684 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:53.936388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:53.942965 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:53.943546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:53.943931 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:53.944558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:53.945004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:53.945417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:54.100618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:54.109696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:54.111428 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:54.112899 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cf0ba38c-d41f-43fb-a2fb-eb5bbced9dd7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.181026 Oct 09 01:02:54.113801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 394/786] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:53 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:55.142492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:55.143137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:55.149585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:55.150206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:55.150965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:55.151778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:55.152306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:55.152844 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:55.323819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:55.334307 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:55.335968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:55.337724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3a29fa7c-4ab7-4ec2-9ebc-56a660be012a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.199396 Oct 09 01:02:55.338863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 393/787] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:55 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:56.277071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-74e1f0f8-e03a-44dc-8487-b2c8e2f8bebc req-1805c194-f579-463b-be6d-bb0a00116b39 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "name": "network-vif-plugged", "status": "completed", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:02:56.292798 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-74e1f0f8-e03a-44dc-8487-b2c8e2f8bebc req-1805c194-f579-463b-be6d-bb0a00116b39 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:56.293803 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-74e1f0f8-e03a-44dc-8487-b2c8e2f8bebc req-1805c194-f579-463b-be6d-bb0a00116b39 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:56.294521 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-74e1f0f8-e03a-44dc-8487-b2c8e2f8bebc req-1805c194-f579-463b-be6d-bb0a00116b39 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:56.310307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-74e1f0f8-e03a-44dc-8487-b2c8e2f8bebc req-1805c194-f579-463b-be6d-bb0a00116b39 service nova] Creating event network-vif-plugged:a48d8008-24c8-488e-9a77-3b3d2c3b71b4 for instance 98a78205-334d-469d-8eb4-26c151df1616 on hv-cinder-maste Oct 09 01:02:56.316833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-74e1f0f8-e03a-44dc-8487-b2c8e2f8bebc req-1805c194-f579-463b-be6d-bb0a00116b39 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044129 Oct 09 01:02:56.317783 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 395/788] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:02:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:02:56.366634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:56.367329 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:56.374137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:56.374894 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:56.375557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:56.376500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:56.376959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:56.377391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:56.549013 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:56.559625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:56.561842 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:56.563445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b6a02d96-082b-4789-ad20-11a4aeac1565 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.203272 Oct 09 01:02:56.584308 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 394/789] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:56 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:57.590730 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:57.592237 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:57.599450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:57.600050 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:57.600696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:57.601429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:57.601888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:57.602318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:57.749110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:57.759816 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:57.761403 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:57.762809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-aba3a537-b03b-4129-afb5-0a3646549177 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.176281 Oct 09 01:02:57.763376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 396/790] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:57 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:02:58.789162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:02:58.794009 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:02:58.804327 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:58.804819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:58.805258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:58.805957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:02:58.806413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:02:58.806862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:02:58.969979 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:58.978905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:58.980388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:02:58.981744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-79a07912-b9e1-49c7-810e-892dc937bf8d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.197086 Oct 09 01:02:58.983626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 395/791] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:02:58 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:00.012132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:00.012846 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:00.019583 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:00.020336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:00.020928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:00.021820 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:00.022454 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:00.022877 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:00.217696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:00.239183 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:00.240864 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:00.242084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0720e72c-f5cf-40b0-8004-6212361bd6ce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.233818 Oct 09 01:03:00.256630 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 397/792] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:00 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:01.272406 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:01.273126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:01.280304 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:01.280887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:01.281334 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:01.282041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:01.282505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:01.282932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:01.423503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:01.432607 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:01.434075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:01.435280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-02b89d7e-0b39-46ae-a12a-8128cde3f3d5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.167703 Oct 09 01:03:01.435880 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 396/793] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:01 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:03:02.466298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:02.467714 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:02.475113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:02.475873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:02.476575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:02.477469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:02.478236 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:02.478738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:02.654885 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:02.665174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:02.670672 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:02.672193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3b21f732-6ad3-4365-b5f9-d7a7df2ee6d7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.211441 Oct 09 01:03:02.672764 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 398/794] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:02 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:03.698907 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:03.699517 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:03.702096 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=27 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:03.708232 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:03.708715 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:03.709150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:03.709837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:03.710292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:03.710715 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:03.714575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=77 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:03.841305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:03.850002 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:03.851435 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:03.852621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-77ba1cad-b6ab-480b-892a-a7a9e59c951e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1741 microversion: 2.20 time: 0.157589 Oct 09 01:03:03.853197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 397/795] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:03 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1741 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:04.878110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:04.878711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:04.885000 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:04.885466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:04.885890 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:04.886558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:04.887022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:04.887438 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:05.014851 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:05.023364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:05.024817 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:05.026198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f96469b7-3dcf-428a-a53c-0e9120af3211 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1723 microversion: 2.20 time: 0.151708 Oct 09 01:03:05.026745 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 399/796] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:04 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:06.051170 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:06.051836 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:06.058209 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:06.058683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:06.059114 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:06.059837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:06.060286 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:06.060707 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:06.192930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:06.201135 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:06.202589 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:06.203782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-34b3489a-405a-493d-a23e-a005ac449940 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1723 microversion: 2.20 time: 0.156460 Oct 09 01:03:06.204346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 398/797] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:06 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:07.230574 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:07.231174 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:07.233571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=29 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:07.238928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:07.240045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:07.240480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:07.241155 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:07.241596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:07.242010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:07.244659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:07.370405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:07.378223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:07.379663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:07.380792 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b3777ac8-2d8c-4a37-9083-6e12c4122d98 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1666 microversion: 2.20 time: 0.154104 Oct 09 01:03:07.381336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 400/798] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:07 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1666 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:07.406239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "78c1d170-966a-471c-9da0-eee1375b2c32"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:07.408650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:07.415605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:07.416095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:07.416532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:07.417215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:07.417673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:07.418095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:07.690062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cf86e68d-f722-463f-a8fe-8d816a7130df tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.287713 Oct 09 01:03:07.690652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 399/799] 10.222.0.33 () {62 vars in 1364 bytes} [Sun Oct 9 01:03:07 2022] POST /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 188 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 01:03:07.813469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:07.814051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:07.820213 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:07.820695 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:07.821125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:07.821808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:07.822255 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:07.822676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:07.961061 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120633) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 09 01:03:08.031092 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-462762b6-8064-43ca-832d-d37a41cbd596 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/action" status: 202 len: 0 microversion: 2.20 time: 0.221367 Oct 09 01:03:08.031789 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 401/800] 10.222.0.33 () {62 vars in 1319 bytes} [Sun Oct 9 01:03:07 2022] POST /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/action => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:03:08.040218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:08.040822 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:08.047344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:08.047862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:08.048304 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:08.049373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:08.049492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:08.049909 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:08.188303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:08.200859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:08.202438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:08.203724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9538f1c7-eede-4003-8279-a22aebc4b383 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1752 microversion: 2.20 time: 0.167759 Oct 09 01:03:08.215712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 400/801] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:08 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1752 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:08.260719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-42554e0e-d85e-4211-9eb6-28bbdf6b61b8 req-333646b6-3e27-48c1-8e8e-355470070d3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "name": "network-vif-unplugged", "status": "completed", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:08.271832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-42554e0e-d85e-4211-9eb6-28bbdf6b61b8 req-333646b6-3e27-48c1-8e8e-355470070d3b service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:08.272497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-42554e0e-d85e-4211-9eb6-28bbdf6b61b8 req-333646b6-3e27-48c1-8e8e-355470070d3b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:08.273126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-42554e0e-d85e-4211-9eb6-28bbdf6b61b8 req-333646b6-3e27-48c1-8e8e-355470070d3b service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:08.286683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.compute.server_external_events [req-42554e0e-d85e-4211-9eb6-28bbdf6b61b8 req-333646b6-3e27-48c1-8e8e-355470070d3b service nova] Unable to find a host for instance 98a78205-334d-469d-8eb4-26c151df1616. Dropping event network-vif-unplugged {{(pid=120633) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 09 01:03:08.287585 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-42554e0e-d85e-4211-9eb6-28bbdf6b61b8 req-333646b6-3e27-48c1-8e8e-355470070d3b service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.031272 Oct 09 01:03:08.288197 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 402/802] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:03:08 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 33 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 09 01:03:09.231052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:09.231698 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:09.238368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:09.238852 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:09.239323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:09.240049 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:09.240501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:09.240924 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:09.371372 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:09.381442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:09.382902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:09.384114 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-dc9da091-8f0a-4321-9771-5d66c0ab75f1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1754 microversion: 2.20 time: 0.157333 Oct 09 01:03:09.384669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 401/803] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:09 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1754 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:10.410723 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:10.411900 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:10.418246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:10.418720 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:10.419146 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:10.419842 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:10.420285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:10.420701 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:10.548595 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:10.557472 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:10.558880 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:10.560038 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c0f4a9bf-0674-45ac-8a03-9fdfdb5f2fc8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1754 microversion: 2.20 time: 0.153173 Oct 09 01:03:10.560589 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 403/804] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:10 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1754 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:11.586066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:11.586593 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:11.593108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:11.593594 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:11.594034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:11.594725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:11.595186 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:11.595641 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:11.721337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:11.729694 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:11.731125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:11.732284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-94f5fc8d-fdac-4f61-b86e-7f1ab6b4330e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1754 microversion: 2.20 time: 0.150703 Oct 09 01:03:11.732837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 402/805] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:11 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1754 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:12.757564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:12.758164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:12.764561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:12.765036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:12.765567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:12.766249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:12.766696 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:12.767115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:12.892612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:12.902220 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:12.903675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:12.904823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c3b34855-8b42-4ced-96bd-bc6a475d9a85 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1754 microversion: 2.20 time: 0.151168 Oct 09 01:03:12.905602 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 404/806] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:12 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1754 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:13.930821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:13.931512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:13.938521 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:13.939000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:13.939438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:13.940156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:13.940602 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:13.941027 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:14.079846 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:14.091912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:14.093761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:14.095017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e7b49b33-5333-40fd-b02c-4b23d25ec57d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1754 microversion: 2.20 time: 0.168774 Oct 09 01:03:14.096441 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 403/807] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:13 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1754 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:15.122202 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:15.122810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:15.128992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:15.129465 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:15.129928 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:15.130603 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:15.131047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:15.131462 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:15.341307 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:15.350285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:15.351831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:15.352985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-57c85dd2-53b1-41f2-b78f-160c9a0e1237 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.234648 Oct 09 01:03:15.353530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 405/808] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:15 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:16.379383 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:16.380082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:16.386899 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:16.388061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:16.388702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:16.389411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:16.389864 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:16.390291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:16.537719 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:16.546288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:16.547753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:16.548908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-28e98f8a-de82-43c4-ba91-3a14211a1da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.173394 Oct 09 01:03:16.549530 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 404/809] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:16 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:17.575354 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:17.575970 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:17.582234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:17.582711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:17.583166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:17.583862 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:17.584310 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:17.584722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:17.725343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:17.735942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:17.737879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:17.739131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c7148209-200e-4b34-baf4-4aeecd9c5d95 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.167610 Oct 09 01:03:17.740497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 406/810] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:17 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:18.764923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:18.766330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:18.773218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:18.773824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:18.774265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:18.774966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:18.775419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:18.776208 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:18.922920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:18.934225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:18.935913 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:18.937163 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-daf500ab-c646-474d-86d3-8bc4a3a3aeba tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.176155 Oct 09 01:03:18.937710 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 405/811] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:18 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:19.962738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:19.963498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:19.970203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:19.970688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:19.971117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:19.971826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:19.972274 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:19.972690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:20.135264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:20.147100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:20.149008 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:20.150456 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c6f9b942-0a7c-46db-b542-fa0f97cceaff tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.192158 Oct 09 01:03:20.152134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 407/812] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:19 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:21.176509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:21.177887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:21.185120 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:21.185926 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:21.186580 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:21.187495 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:21.188201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:21.188863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:21.338142 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:21.347233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:21.349261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:21.350625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9c04407e-34ae-4b5b-8c24-6d6554464201 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.178118 Oct 09 01:03:21.351176 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 406/813] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:21 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:22.377145 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:22.378043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:22.388787 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:22.389273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:22.389705 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:22.390393 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:22.390833 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:22.391249 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:22.535712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:22.549014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:22.550451 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:22.551702 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3ebff4c6-afe0-4809-82f5-f23f3f05fe97 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.178538 Oct 09 01:03:22.552341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 408/814] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:22 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:23.577637 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:23.578270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:23.585153 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:23.585643 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:23.586082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:23.586781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:23.587233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:23.587680 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:23.720545 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:23.729787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:23.731221 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:23.732384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-aad54811-68b0-4a19-a9df-bbc44f18e29b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.158884 Oct 09 01:03:23.732963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 407/815] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:23 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:24.759091 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:24.759688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:24.765991 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:24.766469 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:24.766910 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:24.767615 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:24.768065 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:24.768488 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:24.893271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:24.902389 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:24.903857 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:24.905003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8b658b48-7ef5-46a6-9875-2406e5dddea2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.150541 Oct 09 01:03:24.905558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 409/816] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:24 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:25.929971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:25.930572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:25.937198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:25.937675 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:25.938106 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:25.938783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:25.939224 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:25.939866 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:26.091476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:26.100773 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:26.102362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:26.103405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-73f3f244-a38f-4ef1-9a72-2ed601df22ea tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.177370 Oct 09 01:03:26.104229 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 408/817] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:25 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:27.129033 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:27.129836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:27.136504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:27.137068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:27.137578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:27.138441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:27.138937 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:27.139614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:27.341134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:27.350610 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:27.352088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:27.353252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-98c315ec-5ba0-4c5e-b8f2-5d90c963c582 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.228649 Oct 09 01:03:27.353808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 410/818] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:27 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:28.379859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:28.380473 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:28.386392 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:28.387099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:28.387718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:28.388419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:28.388871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:28.389294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:28.538927 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:28.556997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:28.558892 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:28.560223 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8ac13b2a-ec98-48a8-bc42-5dd77b3b7e2e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.184256 Oct 09 01:03:28.561341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 409/819] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:28 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:29.589468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:29.591100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:29.602670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:29.604140 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:29.604920 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:29.606167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:29.607888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:29.609279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:29.792932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:29.803841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:29.805889 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:29.807750 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7745d0f2-554a-4e52-a73d-fcf2b3789062 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.222992 Oct 09 01:03:29.808653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 411/820] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:29 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:30.836534 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:30.837139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:30.843947 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:30.844431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:30.844863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:30.845624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:30.846080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:30.846504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:31.010075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:31.021611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:31.024748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:31.034683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-10506b0d-8262-4c3c-9888-543080bb2d7b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.194135 Oct 09 01:03:31.048313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 410/821] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:30 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:32.061505 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:32.062090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:32.068312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:32.068782 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:32.069404 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:32.070090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:32.070536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:32.070955 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:32.365472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-895113568", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1094622065", "security_groups": [{"name": "tempest-secgroup-smoke-606774372"}], "block_device_mapping_v2": [{"uuid": "a4eb4fc5-0a95-4f54-9ce1-c868048d1bfd", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "51f66c80-6e94-4ac9-ad00-6dd9246260d1"}]}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:32.512050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.network.neutron [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] validate_networks() for [('51f66c80-6e94-4ac9-ad00-6dd9246260d1', None, None, None, None, None)] {{(pid=120632) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:03:32.656801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:32.667151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:32.669349 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:32.671049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5e20ac90-bd9b-46fa-b6c5-f8d9494987d9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.613395 Oct 09 01:03:32.685303 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 412/822] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:32 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:32.990692 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Flavor limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:03:32.991421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Image limits 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:03:32.992080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Flavor pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:03:32.993712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Image pref 0:0:0 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:03:32.997587 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.virt.hardware [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120632) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:03:33.009385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:33.010132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:33.010736 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:33.012758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:33.014136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:33.014726 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:33.026419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:33.042762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:03:33.048452 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Getting quotas for user 04bc22d558c2466ba43a456d08bdd6bd and project 28bc82345ea24b929b66fbcded2b97b7. Resources: {'cores', 'instances', 'ram'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:03:33.178071 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Going to run 1 instances... {{(pid=120632) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:03:33.180343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a4eb4fc5-0a95-4f54-9ce1-c868048d1bfd',volume_size=None,volume_type=None)] {{(pid=120632) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:03:33.393809 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f3087111-b2fd-4d68-9a43-11e36e375722 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.033029 Oct 09 01:03:33.399694 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 411/823] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:03:32 2022] POST /compute/v2.1/servers => generated 405 bytes in 1040 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:03:33.411940 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b4d85ce4-e22c-442f-9f25-49bb4a6a4456 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:33.412548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b4d85ce4-e22c-442f-9f25-49bb4a6a4456 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:33.496786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b4d85ce4-e22c-442f-9f25-49bb4a6a4456 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:33.503879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b4d85ce4-e22c-442f-9f25-49bb4a6a4456 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1094 microversion: 2.1 time: 0.096431 Oct 09 01:03:33.505907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 413/824] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:33 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1094 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:33.700796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:33.701583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:33.724648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=43,nova_api:INSERT=3 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:33.729730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:33.730272 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:33.730731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:33.731459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:33.734633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:33.735094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:33.960733 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:33.975762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:33.979395 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:33.985845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f30d092d-7840-4375-a481-ae15ed9e81fe tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.290100 Oct 09 01:03:33.986963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 412/825] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:33 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:34.527113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:34.527916 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:34.538020 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:34.539386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:34.540181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:34.541578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:34.543144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:34.543894 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:34.632551 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:34.642901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e807da4b-1c69-4d3d-a024-d0e9453f783d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1192 microversion: 2.1 time: 0.119743 Oct 09 01:03:34.657468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 414/826] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:34 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1192 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:35.015014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:35.015569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:35.025788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:35.031671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:35.032346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:35.033167 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:35.034954 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:35.035623 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:35.246150 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-35732c3a-27b1-4c70-a04e-eb39096686e9 req-4a019534-2df2-42c6-afd2-b38355b79536 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "name": "network-vif-plugged", "status": "completed", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:35.262916 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-35732c3a-27b1-4c70-a04e-eb39096686e9 req-4a019534-2df2-42c6-afd2-b38355b79536 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:35.263835 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-35732c3a-27b1-4c70-a04e-eb39096686e9 req-4a019534-2df2-42c6-afd2-b38355b79536 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:35.267508 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-35732c3a-27b1-4c70-a04e-eb39096686e9 req-4a019534-2df2-42c6-afd2-b38355b79536 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:35.282154 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-35732c3a-27b1-4c70-a04e-eb39096686e9 req-4a019534-2df2-42c6-afd2-b38355b79536 service nova] Creating event network-vif-plugged:a48d8008-24c8-488e-9a77-3b3d2c3b71b4 for instance 98a78205-334d-469d-8eb4-26c151df1616 on hv-cinder-maste Oct 09 01:03:35.294399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-35732c3a-27b1-4c70-a04e-eb39096686e9 req-4a019534-2df2-42c6-afd2-b38355b79536 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052184 Oct 09 01:03:35.295801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 415/827] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:03:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:03:35.474954 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:35.494488 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:35.496235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:35.497560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e45ac317-6df2-499f-b581-93807400aa9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.487650 Oct 09 01:03:35.511616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 413/828] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:35 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:35.678572 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:35.680075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:35.713749 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:35.723158 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:35.723902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:35.730810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:35.732609 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:35.734027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:35.921441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:35.934313 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8bf3f88d-31bd-46f8-ae13-e2d34ff26a8e tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1210 microversion: 2.1 time: 0.268382 Oct 09 01:03:35.936797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 416/829] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:35 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1210 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:36.529713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:36.530893 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:36.538140 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:36.538640 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:36.539082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:36.539835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:36.540295 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:36.540723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:36.706308 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:36.727317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:36.729704 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:36.731834 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-12b81498-ef2e-4b48-8027-fae1050808f7 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.206262 Oct 09 01:03:36.733302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 414/830] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:36 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:36.971004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:36.974078 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:36.980493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:36.981391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:36.982266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:36.983234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:36.984089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:36.984714 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:37.213423 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:37.222379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cc82b339-dd9f-46aa-9f72-299d9628d2be tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.264587 Oct 09 01:03:37.224360 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 417/831] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:36 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:37.389373 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-9995bf54-b9ba-4626-a8c4-d9e7972d53d5 req-dfc550b7-938f-463f-8f45-60d18071d9e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e8b0e22-9d18-412f-97b9-bf5e750c9dcb", "tag": "07d4f0c5-1202-4554-b832-08bb967ca163"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:37.400333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-9995bf54-b9ba-4626-a8c4-d9e7972d53d5 req-dfc550b7-938f-463f-8f45-60d18071d9e4 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:37.400756 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-9995bf54-b9ba-4626-a8c4-d9e7972d53d5 req-dfc550b7-938f-463f-8f45-60d18071d9e4 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:37.401722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-9995bf54-b9ba-4626-a8c4-d9e7972d53d5 req-dfc550b7-938f-463f-8f45-60d18071d9e4 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:37.414960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-9995bf54-b9ba-4626-a8c4-d9e7972d53d5 req-dfc550b7-938f-463f-8f45-60d18071d9e4 service nova] Creating event network-changed:07d4f0c5-1202-4554-b832-08bb967ca163 for instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb on hv-cinder-maste Oct 09 01:03:37.420854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-9995bf54-b9ba-4626-a8c4-d9e7972d53d5 req-dfc550b7-938f-463f-8f45-60d18071d9e4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035131 Oct 09 01:03:37.422149 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 415/832] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:03:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:03:37.762081 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:37.762946 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:37.771019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:37.773001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:37.773677 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:37.774375 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:37.775428 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:37.776654 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:37.777237 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:37.783934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:38.090391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:38.106594 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:38.109055 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:38.115288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bec7b557-7185-46d5-82e4-285b2280a68b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1810 microversion: 2.20 time: 0.359489 Oct 09 01:03:38.122522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 418/833] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:37 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1810 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:03:38.245962 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:38.246662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:38.257836 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:38.258370 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:38.258692 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:38.260971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:38.261439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:38.261857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:38.522529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:38.531055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-96349c07-c930-45c0-8bf2-b041952022c5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.289145 Oct 09 01:03:38.533084 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 416/834] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:38 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:39.155075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:39.156486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:39.163226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:39.163737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:39.164175 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:39.164896 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:39.167473 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:39.167948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:39.369340 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:39.380699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:39.383104 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:39.385022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bb096271-9381-4834-851f-ff712b808d8f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1797 microversion: 2.20 time: 0.233778 Oct 09 01:03:39.386618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 419/835] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:03:39 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1797 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:03:39.558560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:39.559069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:39.566093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:39.570794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:39.571305 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:39.572082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:39.576057 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:39.578509 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:39.732130 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:39.740888 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-bd03d75e-50a3-429a-9003-0ccdd8ee2a18 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.187629 Oct 09 01:03:39.742703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 417/836] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:39 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:40.774409 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:40.775697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:40.784166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:40.784681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:40.785317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:40.786926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:40.787705 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:40.788288 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:40.932303 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:40.944127 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-567ea695-056b-4fec-b304-68a708d41e1b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.181210 Oct 09 01:03:40.944995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 420/837] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:40 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:41.967863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:41.969189 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:41.977940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:41.978731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:41.979417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:41.980391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:41.981067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:41.981725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:42.191683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:42.201742 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-53085117-9220-497a-b2ea-10223415310a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.238260 Oct 09 01:03:42.202323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 418/838] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:41 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:43.227515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:43.228166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:43.249444 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:43.250203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:43.250853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:43.251564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:43.252041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:43.252463 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:43.425248 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:43.434554 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-75e5f245-1d9d-4d65-9850-b7539ec37f02 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.210829 Oct 09 01:03:43.435461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 421/839] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:43 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:44.459496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:44.460781 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:44.463445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:44.464488 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:44.473868 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:44.474638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:44.475250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:44.476141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:44.476752 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:44.477352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:44.617222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:44.625050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1390554f-54e9-4a2b-8a78-2ef64b126c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.170592 Oct 09 01:03:44.639976 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 419/840] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:44 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:45.649622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:45.650840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:45.653043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:45.658901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:45.659396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:45.659858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:45.660725 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:45.661185 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:45.661750 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:45.799408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:45.808108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c700f5a7-8bd8-4ebc-a70c-c4ec20a55ab8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.163868 Oct 09 01:03:45.808888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 422/841] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:45 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:46.830291 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:46.830895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:46.838171 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:46.838659 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:46.839097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:46.839837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:46.840296 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:46.840721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:46.907040 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0ded1960-07dc-4924-b44a-ba00fcdc828b tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Action: 'create', calling method: .version_select at 0x7f7268a7b8b0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-327703906", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB7RyHq4jihRg3930F7JI3lm3RCi6RgjfaosRMxu2lNVZAyAPJDd9vjD2FrheYOT8Z/07Uh4YvOxvvLgodcIU9fjXuTxOOmaN+djWMtYjOWIU4z7yeihyAcRxXyXvM3KHA=="}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:46.922600 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-0ded1960-07dc-4924-b44a-ba00fcdc828b tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Getting quotas for project f37de57a22a246f0935f85392898dd42. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:03:46.926480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-0ded1960-07dc-4924-b44a-ba00fcdc828b tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Getting quotas for user 3650b0b2b7014e0f8a7a5af9ce92374a and project f37de57a22a246f0935f85392898dd42. Resources: {'key_pairs'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:03:46.950481 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0ded1960-07dc-4924-b44a-ba00fcdc828b tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.047488 Oct 09 01:03:46.951066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 423/842] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 01:03:46 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:03:46.977151 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:46.984901 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-1b266ceb-997d-4b46-b7e5-cb5c85e15a05 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.158372 Oct 09 01:03:46.985494 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 420/843] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:46 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:47.000969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-893014760", "imageRef": "39847b98-3df5-4177-bd6d-7a6212b2d168", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-327703906", "networks": [{"uuid": "6785419e-1d43-419a-bc35-ef45112ab443"}]}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:47.052697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] validate_networks() for [('6785419e-1d43-419a-bc35-ef45112ab443', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:03:47.415505 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:03:47.415974 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:03:47.416430 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:03:47.416838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:03:47.417246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:03:47.425891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:47.426881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:47.427313 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:47.428054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:47.428506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:47.428927 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:47.445711 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Getting quotas for project f37de57a22a246f0935f85392898dd42. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:03:47.449657 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Getting quotas for user 3650b0b2b7014e0f8a7a5af9ce92374a and project f37de57a22a246f0935f85392898dd42. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:03:47.460151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:03:47.462304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='39847b98-3df5-4177-bd6d-7a6212b2d168',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:03:47.506930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4b242df7-7062-42a0-ac87-7d2b9b5edb7d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.509756 Oct 09 01:03:47.507803 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 424/844] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:03:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 511 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:03:47.518496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-24e9553f-23f6-4ad5-8ee0-231999d63d01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:47.519134 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-24e9553f-23f6-4ad5-8ee0-231999d63d01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:47.573755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-24e9553f-23f6-4ad5-8ee0-231999d63d01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:47.580808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-24e9553f-23f6-4ad5-8ee0-231999d63d01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1257 microversion: 2.1 time: 0.066171 Oct 09 01:03:47.581366 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 421/845] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:47 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1257 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:48.008151 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:48.008737 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:48.014654 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:48.015137 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:48.015574 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:48.016277 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:48.016729 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:48.017149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:48.153260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:48.161661 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-047a8017-ed1a-4c48-91f5-df744eab2c56 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.158192 Oct 09 01:03:48.162243 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 425/846] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:48 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:48.603902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:48.604511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:48.611479 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:48.611991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:48.612609 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:48.613169 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:48.614313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:48.614787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:48.675510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:48.683761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-75269e6a-1860-44ef-9874-e79560b7528a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1309 microversion: 2.1 time: 0.083612 Oct 09 01:03:48.684321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 422/847] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:48 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1309 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:03:49.184716 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:49.185325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:49.208475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:49.209301 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:49.214852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:49.215968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:49.216646 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:49.217264 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:49.502746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:49.511072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-92c3ac4e-0fc5-4b36-8ffe-fe3bc60c2cf1 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.330757 Oct 09 01:03:49.511843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 426/848] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:49 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:49.706391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:49.706995 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:49.713319 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:49.713795 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:49.714227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:49.714932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:49.715571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:49.716100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:49.891110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:49.908242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f53cfc6b-8c25-4cd0-8dd2-d50519fcfb31 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.205391 Oct 09 01:03:49.909218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 423/849] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:49 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:50.517158 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-cab1643e-3304-4ebf-a377-ab4c48b8134d req-e853195e-d0bd-43f5-94cc-cf06c129b3f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e8b0e22-9d18-412f-97b9-bf5e750c9dcb", "name": "network-vif-plugged", "status": "completed", "tag": "07d4f0c5-1202-4554-b832-08bb967ca163"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:50.527506 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cab1643e-3304-4ebf-a377-ab4c48b8134d req-e853195e-d0bd-43f5-94cc-cf06c129b3f4 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:50.528075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cab1643e-3304-4ebf-a377-ab4c48b8134d req-e853195e-d0bd-43f5-94cc-cf06c129b3f4 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:50.528503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-cab1643e-3304-4ebf-a377-ab4c48b8134d req-e853195e-d0bd-43f5-94cc-cf06c129b3f4 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:50.534658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:50.535283 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:50.541448 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:50.542178 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:50.542766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:50.543626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:50.544232 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:50.544821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:50.552144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-cab1643e-3304-4ebf-a377-ab4c48b8134d req-e853195e-d0bd-43f5-94cc-cf06c129b3f4 service nova] Creating event network-vif-plugged:07d4f0c5-1202-4554-b832-08bb967ca163 for instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb on hv-cinder-maste Oct 09 01:03:50.557957 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-cab1643e-3304-4ebf-a377-ab4c48b8134d req-e853195e-d0bd-43f5-94cc-cf06c129b3f4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044474 Oct 09 01:03:50.559238 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 427/850] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:03:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:03:50.700679 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:50.708765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-56327c49-a1ad-4750-831a-c8ce16b8b8d2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.178532 Oct 09 01:03:50.717351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 424/851] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:50 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:03:50.932847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:50.933996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:50.940683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:50.941412 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:50.943724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:50.944440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:50.944893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:50.945312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:51.091063 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:51.098710 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8be6b6f2-d3d4-430e-b2c7-c8993e89b3d9 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.170098 Oct 09 01:03:51.099287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 428/852] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:50 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:51.731857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:51.733100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:51.740912 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:51.741477 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:51.742065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:51.742989 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:51.743526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:51.744089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:51.888558 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:51.897270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5551d853-5aa8-495a-aa91-c651a45679b3 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.169197 Oct 09 01:03:51.897713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 425/853] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:51 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:52.121830 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:52.122564 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:52.129596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:52.130079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:52.130704 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:52.131700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:52.132443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:52.133025 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:52.279530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:52.287274 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b4b91abd-0488-4444-a0e1-c5c9eb9814db tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.169358 Oct 09 01:03:52.287887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 429/854] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:52 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:52.920459 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:52.921063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:52.936205 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:52.936703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:52.937341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:52.938057 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:52.938519 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:52.938952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:53.089674 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:53.099060 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c8a77cff-e2c4-4f51-a8a6-33337461af3b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.182351 Oct 09 01:03:53.100028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 426/855] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:52 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:53.310437 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:53.311308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:53.318449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:53.319008 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:53.319513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:53.320303 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:53.320821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:53.321346 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:53.465838 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:53.473477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cea91309-aa5d-4086-a488-865749429daa tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.166978 Oct 09 01:03:53.474047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 430/856] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:53 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:54.126384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:54.126999 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:54.134035 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:54.134523 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:54.134964 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:54.135691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:54.136155 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:54.136579 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:54.274895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:54.285062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ed694bb8-5180-46ed-8e4b-89a4412de5c2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.162384 Oct 09 01:03:54.287032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 427/857] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:54 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:54.497304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:54.497958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:54.505192 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:54.505732 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:54.506171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:54.508914 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:54.509391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:54.509822 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:54.646497 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:54.657066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f03306e5-44fa-4c8b-8a55-867cc6e698ae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.163692 Oct 09 01:03:54.658053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 431/858] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:54 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:55.308529 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:55.309183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:55.315614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:55.316179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:55.316651 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:55.317343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:55.317802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:55.318232 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:55.474292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:55.482340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2935c97f-3379-4f72-afa4-8e8112c193a0 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.177697 Oct 09 01:03:55.482895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 428/859] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:55 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:55.680195 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:55.682143 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:55.688694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:55.689196 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:55.689702 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:55.690535 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:55.691316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:55.691825 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:55.838583 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:55.846667 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-efa92572-a4e5-447e-9943-2f3d8c1fc4cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.170446 Oct 09 01:03:55.847245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 432/860] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:55 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:56.507046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:56.507738 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:56.514210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:56.514690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:56.515129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:56.516041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:56.516503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:56.517174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:56.870944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:56.871563 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:56.878147 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:56.878649 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:56.884186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:56.884186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:56.884186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:56.884186 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:56.884639 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:56.890621 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8c939fd7-24c4-412c-a0ba-6cce2482f854 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.387632 Oct 09 01:03:56.895846 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 429/861] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:56 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:57.037293 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:57.045139 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e958e134-b3fe-4612-95db-55d31c91c594 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.178107 Oct 09 01:03:57.045702 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 433/862] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:56 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:03:57.914578 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:57.915411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:57.917493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:57.923367 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:57.924270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:57.924413 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:57.925095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:57.926343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:57.926751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:57.932455 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=100 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:58.061316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:58.068117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:58.068720 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:58.070627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:58.071282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=10 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:03:58.080409 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:58.080881 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:58.081303 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:58.082039 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:58.083091 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:58.083528 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:58.092312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-3aba09eb-d3d7-4895-b552-75611a4d2124 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.181476 Oct 09 01:03:58.092896 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 430/863] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:57 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:03:58.289455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:58.297153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-91a700c0-2dcb-4870-bdd0-293a2730d7cd tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.233369 Oct 09 01:03:58.297717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 434/864] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:58 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:03:58.706265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-a00788e3-e7ce-4e7f-8605-7300ac91a1f6 req-5569dd2e-e008-48ae-b7d2-bd695053dfbf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:03:58.715618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-a00788e3-e7ce-4e7f-8605-7300ac91a1f6 req-5569dd2e-e008-48ae-b7d2-bd695053dfbf service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:58.716041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-a00788e3-e7ce-4e7f-8605-7300ac91a1f6 req-5569dd2e-e008-48ae-b7d2-bd695053dfbf service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:58.716515 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-a00788e3-e7ce-4e7f-8605-7300ac91a1f6 req-5569dd2e-e008-48ae-b7d2-bd695053dfbf service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:58.727357 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-a00788e3-e7ce-4e7f-8605-7300ac91a1f6 req-5569dd2e-e008-48ae-b7d2-bd695053dfbf service nova] Creating event network-changed:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:03:58.732309 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-a00788e3-e7ce-4e7f-8605-7300ac91a1f6 req-5569dd2e-e008-48ae-b7d2-bd695053dfbf service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029716 Oct 09 01:03:58.732895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 431/865] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:03:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:03:59.115320 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:59.116010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:59.122301 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:59.122779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:59.123212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:59.123932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:59.124383 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:59.124807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:59.291672 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:59.299177 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5cdfe2a5-6da4-4dba-b80f-a71d19bf1b5f tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1265 microversion: 2.1 time: 0.187711 Oct 09 01:03:59.299754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 435/866] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:59 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1265 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:03:59.321100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:03:59.321785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:03:59.327961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:59.328438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:59.328864 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:59.329574 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:03:59.330019 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:03:59.330437 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:03:59.462340 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:03:59.469960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2324dafd-261c-4ffa-847a-a20b96d49fd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.153365 Oct 09 01:03:59.470526 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 432/867] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:03:59 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:00.321969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:00.322575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:00.328471 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:00.328954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:00.329390 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:00.330071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:00.330543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:00.330966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:00.493355 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:00.493937 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:00.500073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:00.500554 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:00.500990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:00.501685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:00.502137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:00.502556 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:00.538722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:00.546554 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9364c197-f27a-400a-ad0e-c068186a58a2 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1453 microversion: 2.1 time: 0.228515 Oct 09 01:04:00.547132 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 436/868] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:00 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1453 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:00.569144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:00.570128 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:00.576149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:00.576911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:00.577343 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:00.578026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:00.578679 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:00.579097 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:00.583184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=100 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:00.638531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:00.646361 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-4b8dc345-bcad-4695-b2cd-83eab958c75d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.156876 Oct 09 01:04:00.646932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 433/869] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:00 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:04:00.885613 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:00.893442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3d84f77d-d67f-4b84-904c-eae0ec6d90ba tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1453 microversion: 2.1 time: 0.328103 Oct 09 01:04:00.894011 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 437/870] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:00 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1453 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:04:01.669066 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:01.669677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:01.676132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:01.676622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:01.677062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:01.677971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:01.678461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:01.678904 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:01.880693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:01.888315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-15d2d0cf-b696-48b4-94c8-85afb3ec8ffb tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.223036 Oct 09 01:04:01.889185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 434/871] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:01 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:02.555416 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-7b27be23-da6b-4836-845b-257f4423ab0a req-a44e1a11-e7eb-4951-b04a-b37825b6f584 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e8b0e22-9d18-412f-97b9-bf5e750c9dcb", "tag": "07d4f0c5-1202-4554-b832-08bb967ca163"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:02.565533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7b27be23-da6b-4836-845b-257f4423ab0a req-a44e1a11-e7eb-4951-b04a-b37825b6f584 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:02.566008 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7b27be23-da6b-4836-845b-257f4423ab0a req-a44e1a11-e7eb-4951-b04a-b37825b6f584 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:02.566468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7b27be23-da6b-4836-845b-257f4423ab0a req-a44e1a11-e7eb-4951-b04a-b37825b6f584 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:02.577961 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-7b27be23-da6b-4836-845b-257f4423ab0a req-a44e1a11-e7eb-4951-b04a-b37825b6f584 service nova] Creating event network-changed:07d4f0c5-1202-4554-b832-08bb967ca163 for instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb on hv-cinder-maste Oct 09 01:04:02.583516 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-7b27be23-da6b-4836-845b-257f4423ab0a req-a44e1a11-e7eb-4951-b04a-b37825b6f584 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031969 Oct 09 01:04:02.587761 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 438/872] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:02.914006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:02.915199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:02.923544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:02.924067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:02.924508 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:02.925241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:02.926028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:02.926629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:03.106904 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:03.115323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b8d456d7-ea26-48bc-96d8-a2d5a646ad0c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.206164 Oct 09 01:04:03.115935 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 435/873] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:02 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:04.139267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:04.139986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:04.146752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:04.147225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:04.147683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:04.148365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:04.148809 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:04.149223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:04.280023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:04.287429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8619d324-0d0e-41f5-bb3b-f3d1e326a3ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.152853 Oct 09 01:04:04.288017 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 439/874] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:04 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:05.310258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:05.310803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:05.313284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=50 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:05.319377 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:05.319874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:05.320311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:05.320997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:05.321451 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:05.321871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:05.449185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:05.456737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-14800135-73a6-48d8-b1ec-55e9eb6051bf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.150387 Oct 09 01:04:05.457300 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 436/875] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:05 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:06.478553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:06.479160 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:06.486003 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:06.486486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:06.486915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:06.487617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:06.488062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:06.488479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:06.590270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-7f064906-6a09-4ff9-bb1b-4349fb2e7de4 req-ecead5dc-b9b9-4c4f-ab04-e2021b860562 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "name": "network-vif-plugged", "status": "completed", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:06.602633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-7f064906-6a09-4ff9-bb1b-4349fb2e7de4 req-ecead5dc-b9b9-4c4f-ab04-e2021b860562 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:06.603257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-7f064906-6a09-4ff9-bb1b-4349fb2e7de4 req-ecead5dc-b9b9-4c4f-ab04-e2021b860562 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:06.603544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-7f064906-6a09-4ff9-bb1b-4349fb2e7de4 req-ecead5dc-b9b9-4c4f-ab04-e2021b860562 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:06.619672 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-7f064906-6a09-4ff9-bb1b-4349fb2e7de4 req-ecead5dc-b9b9-4c4f-ab04-e2021b860562 service nova] Creating event network-vif-plugged:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:04:06.621376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:06.625016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-7f064906-6a09-4ff9-bb1b-4349fb2e7de4 req-ecead5dc-b9b9-4c4f-ab04-e2021b860562 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038875 Oct 09 01:04:06.626225 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 437/876] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:06.630304 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e0e95698-f19b-43e8-a3e2-acecaf391686 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.155685 Oct 09 01:04:06.631045 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 440/877] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:06 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:07.654848 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:07.655611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:07.662348 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:07.662827 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:07.663262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:07.663968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:07.664421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:07.664843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:07.791648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:07.799253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-69d5c6dd-a63e-4d10-b607-9e57d5c0e364 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1357 microversion: 2.1 time: 0.149304 Oct 09 01:04:07.799843 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 438/878] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:07 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1357 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:08.822396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:08.822925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:08.825430 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=4 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:08.830802 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:08.831319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:08.831788 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:08.833066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:08.833516 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:08.833940 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:08.961062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:08.968623 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9da2b15f-483e-4afb-8dfe-8f289520693d tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1548 microversion: 2.1 time: 0.150130 Oct 09 01:04:08.969188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 441/879] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:08 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1548 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:08.991268 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:08.991910 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:08.998342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:08.998826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:08.999265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:09.000014 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:09.000470 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:09.000897 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:09.127924 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:09.136055 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a676f65c-6c1a-46c5-82fc-43a0b946a2e1 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1548 microversion: 2.1 time: 0.149405 Oct 09 01:04:09.136610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 439/880] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:08 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1548 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:09.157622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:09.162411 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:09.163159 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Searching by: {'deleted': False, 'project_id': 'f37de57a22a246f0935f85392898dd42'} {{(pid=120633) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 09 01:04:09.169565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:09.170032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:09.170460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:09.171166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:09.171626 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:09.172050 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:09.260261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.multi_cell_list [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120633) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 09 01:04:09.503870 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8d6bc907-68fe-4051-a253-b0ef48e867ee tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.349870 Oct 09 01:04:09.504472 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 442/881] 10.222.0.33 () {58 vars in 1125 bytes} [Sun Oct 9 01:04:09 2022] GET /compute/v2.1/servers => generated 342 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:09.516198 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:09.516803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:09.522782 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:09.523258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:09.523725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:09.524409 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:09.524859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:09.525282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:09.656683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:09.664234 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-05f0f53a-4573-4f28-909b-c4954dae0721 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1548 microversion: 2.1 time: 0.151957 Oct 09 01:04:09.664791 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 440/882] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:09 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1548 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:11.225886 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c0244eb3-4652-4f88-b2c3-e7478d390ae2"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:11.228470 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:11.234850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:11.235316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:11.235780 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:11.236458 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:11.236903 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:11.237316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:11.511179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c613cdb3-b809-4c3e-bb62-8605df2b7983 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.289605 Oct 09 01:04:11.511802 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 443/883] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:04:11 2022] POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments => generated 194 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:12.575426 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:12.575982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:12.583107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:12.583609 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:12.584062 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:12.584757 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:12.585220 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:12.585634 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:12.614041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-85ab5c54-cb40-44ba-ae56-9933e0f5de3b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments/78c1d170-966a-471c-9da0-eee1375b2c32" status: 200 len: 194 microversion: 2.20 time: 0.042691 Oct 09 01:04:12.614618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 441/884] 10.222.0.33 () {60 vars in 1454 bytes} [Sun Oct 9 01:04:12 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments/78c1d170-966a-471c-9da0-eee1375b2c32 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:12.684096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:12.684664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:12.690653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:12.691124 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:12.692085 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:12.692758 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:12.693200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:12.693613 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:12.771315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lazy-loading 'services' on Instance uuid 98a78205-334d-469d-8eb4-26c151df1616 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:04:12.840503 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-428b8d46-987b-4873-93be-81cbabcffaaf tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments/78c1d170-966a-471c-9da0-eee1375b2c32" status: 202 len: 0 microversion: 2.20 time: 0.161014 Oct 09 01:04:12.841088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 444/885] 10.222.0.33 () {60 vars in 1457 bytes} [Sun Oct 9 01:04:12 2022] DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments/78c1d170-966a-471c-9da0-eee1375b2c32 => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 09 01:04:12.849405 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:12.851791 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:12.858285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:12.858869 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:12.859323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:12.860161 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:12.860661 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:12.861051 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:12.891924 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-72d0e753-a0e7-4dbf-8fc4-ec6be6854189 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046390 Oct 09 01:04:12.892482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 442/886] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:12 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:13.905474 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:13.907717 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:13.913871 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:13.914353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:13.914786 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:13.915468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:13.915954 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:13.916379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:13.945424 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e8da3936-1ee5-4517-88fb-1af250a02cca tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045651 Oct 09 01:04:13.945962 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 445/887] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:13 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:14.958202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:14.960438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:14.967189 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:14.967727 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:14.968157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:14.968830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:14.969274 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:14.969691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:14.999539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-90a1125a-559d-4f04-bab7-a1828d42ac42 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045381 Oct 09 01:04:15.000094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 443/888] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:14 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 09 01:04:16.012418 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:16.014582 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:16.020978 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:16.021465 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:16.021904 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:16.022590 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:16.023149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:16.023581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:16.052963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-85aefc23-33a7-4a01-8ae4-2bd594da0c1f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044621 Oct 09 01:04:16.053511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 446/889] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:16 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:17.065491 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:17.068083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:17.074571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:17.075052 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:17.075484 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:17.076193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:17.076642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:17.077125 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:17.107312 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-33aa3bf0-29a0-4adf-b600-0980dee33da2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045788 Oct 09 01:04:17.107902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 444/890] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:17 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:18.124426 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:18.127764 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:18.134939 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:18.135415 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:18.135885 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:18.136571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:18.137016 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:18.137434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:18.169096 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e137a6d7-b588-495c-9ac7-9557b72b1682 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051939 Oct 09 01:04:18.169627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 447/891] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:18 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:19.100569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:19.101161 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:19.108103 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:19.108586 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:19.109023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:19.109767 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:19.110207 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:19.110657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:19.181560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:19.183713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:19.191919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:19.192445 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:19.192877 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:19.193941 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:19.194041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:19.194996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:19.210515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:19.236425 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-591831d5-8fb0-4b4d-b349-bb356dbdac52 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058844 Oct 09 01:04:19.237001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 448/892] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:19 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:19.250093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:19.258390 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-27a3c086-fa7c-4d62-9144-60ee250679cf tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1594 microversion: 2.1 time: 0.162083 Oct 09 01:04:19.259050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 445/893] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:19 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1594 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:20.249841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:20.251960 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:20.258728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:20.259213 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:20.259671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:20.260405 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:20.260850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:20.261267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:20.289858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-28068bc8-36db-458e-a1ed-5e541a840cd5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.044122 Oct 09 01:04:20.290402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 449/894] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:04:20 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:20.355023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:20.355569 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:20.361873 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:20.362341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:20.362774 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:20.364027 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:20.364480 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:20.364908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:20.392063 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:04:20.437272 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:20.468737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a13506c3-8b60-4d8f-8ea3-f2d233fef3bd tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 204 len: 0 microversion: 2.20 time: 0.117589 Oct 09 01:04:20.469357 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 446/895] 10.222.0.33 () {60 vars in 1280 bytes} [Sun Oct 9 01:04:20 2022] DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 09 01:04:20.478425 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:20.479248 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:20.485150 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:20.485707 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:20.486207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:20.486969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:20.487467 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:20.487911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:20.619834 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:20.628130 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:20.629635 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:20.630795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ec0404b5-a882-4f46-b098-fbe9cc9bdf5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.156456 Oct 09 01:04:20.631348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 450/896] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:20 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:04:21.165020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-4ac8ab01-a19f-4ff7-9671-a017eed278a0 req-9ccd93af-f884-452a-b15a-3a9f93f617ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:21.174338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-4ac8ab01-a19f-4ff7-9671-a017eed278a0 req-9ccd93af-f884-452a-b15a-3a9f93f617ba service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:21.174812 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-4ac8ab01-a19f-4ff7-9671-a017eed278a0 req-9ccd93af-f884-452a-b15a-3a9f93f617ba service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:21.175243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-4ac8ab01-a19f-4ff7-9671-a017eed278a0 req-9ccd93af-f884-452a-b15a-3a9f93f617ba service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:21.187275 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-4ac8ab01-a19f-4ff7-9671-a017eed278a0 req-9ccd93af-f884-452a-b15a-3a9f93f617ba service nova] Creating event network-changed:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:04:21.192264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-4ac8ab01-a19f-4ff7-9671-a017eed278a0 req-9ccd93af-f884-452a-b15a-3a9f93f617ba service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031742 Oct 09 01:04:21.193051 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 447/897] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:21.656813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:21.657700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:21.665198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:21.665848 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:21.666392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:21.667212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:21.667820 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:21.668473 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:21.960306 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:21.970008 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:21.971722 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:21.973461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7049b215-535b-455e-9cfc-004834004bb3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.320621 Oct 09 01:04:21.974004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 451/898] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:21 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:22.466990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1915066342"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:22.467683 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:22.473717 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:22.474195 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:22.474626 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:22.475303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:22.475778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:22.476194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:22.606077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.network.security_group_api [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Adding security group dc056f2c-4cfe-4712-a0d3-62e45f5a9644 to port ebfdac77-d547-424a-bd77-91436ba7750c Oct 09 01:04:22.952722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-497dcf61-9f4f-4c2a-b92d-6b8d8f78ab8e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/action" status: 202 len: 0 microversion: 2.1 time: 0.490546 Oct 09 01:04:22.954041 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 448/899] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 01:04:22 2022] POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/action => generated 0 bytes in 493 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:04:22.963878 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:22.964491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:22.971311 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:22.972021 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:22.972491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:22.973189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:22.973642 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:22.974060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:22.981117 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:23.000758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:23.001808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:23.014682 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:23.015854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:23.017725 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:23.027095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:23.027858 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:23.028998 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:23.178332 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:23.188148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:23.189743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:23.190918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-640b6752-73b9-4c63-9703-f21ff771e197 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1725 microversion: 2.20 time: 0.194436 Oct 09 01:04:23.192037 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 449/900] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:22 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1725 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:23.211191 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-6bcbf378-fbaa-4fc6-ad49-d0101d92f1d0 req-c9849cc1-231d-4e9e-9048-9250fdac3d96 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:23.223631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-6bcbf378-fbaa-4fc6-ad49-d0101d92f1d0 req-c9849cc1-231d-4e9e-9048-9250fdac3d96 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:23.224650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-6bcbf378-fbaa-4fc6-ad49-d0101d92f1d0 req-c9849cc1-231d-4e9e-9048-9250fdac3d96 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:23.225259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-6bcbf378-fbaa-4fc6-ad49-d0101d92f1d0 req-c9849cc1-231d-4e9e-9048-9250fdac3d96 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:23.241935 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-6bcbf378-fbaa-4fc6-ad49-d0101d92f1d0 req-c9849cc1-231d-4e9e-9048-9250fdac3d96 service nova] Creating event network-changed:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:04:23.248602 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-6bcbf378-fbaa-4fc6-ad49-d0101d92f1d0 req-c9849cc1-231d-4e9e-9048-9250fdac3d96 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040996 Oct 09 01:04:23.254544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 450/901] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:23.316787 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:23.326286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-50502e55-ba15-4b3b-898c-4fec0a0c41b8 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1641 microversion: 2.1 time: 0.367309 Oct 09 01:04:23.327051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 452/902] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:22 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1641 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:23.997528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-54be4ada-5f57-42fa-b831-1a34b63cfd0b req-9b21d9be-58c4-4e09-bc35-8d4cf4cccbf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "98a78205-334d-469d-8eb4-26c151df1616", "name": "network-vif-deleted", "tag": "a48d8008-24c8-488e-9a77-3b3d2c3b71b4"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:24.023567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-54be4ada-5f57-42fa-b831-1a34b63cfd0b req-9b21d9be-58c4-4e09-bc35-8d4cf4cccbf9 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:24.025051 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-54be4ada-5f57-42fa-b831-1a34b63cfd0b req-9b21d9be-58c4-4e09-bc35-8d4cf4cccbf9 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:24.025673 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-54be4ada-5f57-42fa-b831-1a34b63cfd0b req-9b21d9be-58c4-4e09-bc35-8d4cf4cccbf9 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:24.041270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-54be4ada-5f57-42fa-b831-1a34b63cfd0b req-9b21d9be-58c4-4e09-bc35-8d4cf4cccbf9 service nova] Creating event network-vif-deleted:a48d8008-24c8-488e-9a77-3b3d2c3b71b4 for instance 98a78205-334d-469d-8eb4-26c151df1616 on hv-cinder-maste Oct 09 01:04:24.047023 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-54be4ada-5f57-42fa-b831-1a34b63cfd0b req-9b21d9be-58c4-4e09-bc35-8d4cf4cccbf9 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056295 Oct 09 01:04:24.047672 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 451/903] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:24.218240 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:24.221062 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:24.228217 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:24.228712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:24.229182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:24.229882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:24.230339 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:24.230910 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:24.264369 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:24.264858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:24.265298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:24.265753 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lazy-loading 'fault' on Instance uuid 98a78205-334d-469d-8eb4-26c151df1616 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:04:24.312319 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:24.321342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:24.322852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:24.323959 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6fbf9350-1031-4cd9-b3c9-8fc885fb09d1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 200 len: 1372 microversion: 2.20 time: 0.109617 Oct 09 01:04:24.324875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 453/904] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:24 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 1372 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:25.350650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:25.351258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:25.358141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:25.358618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:25.359049 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:25.359753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:25.360201 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:25.360619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:25.374219 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] HTTP exception thrown: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Request method failure captured: Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 HTTP/1.1 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh1zN0RFejgIlRejGebeuKod3w-XJOgFtBs1rs2AyaEyhIOmlH5SPOmzYiX14QlhxrykcEOZPGPR3baTn6hCjNBhJl6gEE4NxbzyP8nsobszHsmCZ9Ixv6cq69LQ6LeEoSfpt5SjWL79E30Tz4Y9e8xRrsDjNijB2ku6Nq_M4qVItyk Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Openstack-Nova-Api-Version: 2.20 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:04:25.375828 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 56b5d764f0044358ac594a2edd6f4c0f Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:04:25.377544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '98a78205-334d-469d-8eb4-26c151df1616'} Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:04:25.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:04:25.380139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:04:25.381800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Returning 404 to user: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:04:25.381800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-eae64fe2-08fc-4db8-bba0-4e241c52a5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 404 len: 111 microversion: 2.20 time: 0.031690 Oct 09 01:04:25.381800 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 452/905] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:25 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:04:25.386044 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a5e1f904-2756-4649-99f9-8d13c15d28fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '.version_select at 0x7f7268a4c310>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:25.404698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a5e1f904-2756-4649-99f9-8d13c15d28fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1083022739" status: 204 len: 0 microversion: 2.20 time: 0.022449 Oct 09 01:04:25.405285 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 454/906] 10.222.0.33 () {60 vars in 1262 bytes} [Sun Oct 9 01:04:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1083022739 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 09 01:04:26.027755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-80ee0a61-a4be-47ed-9f4c-9e1db2e8246c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'create', calling method: .version_select at 0x7f7268b988b0>, body: {"keypair": {"name": "tempest-keypair-454303479", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM3Eb1EslbODWphvyyL4ge2S4/ryrWs+FwE3wZ7AlyKM450Ac5e00CaUgBaV2gz+UcGhmFTwCUc4EuclEdbbdaYetip71HNucZz/uzFgulI1d8HQ1zdKM4AviP+9VkEtVA=="}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:26.043454 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-80ee0a61-a4be-47ed-9f4c-9e1db2e8246c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for project 410212da4a794c07b30ed22cc5b7e760. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:04:26.047239 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-80ee0a61-a4be-47ed-9f4c-9e1db2e8246c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for user 56b5d764f0044358ac594a2edd6f4c0f and project 410212da4a794c07b30ed22cc5b7e760. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:04:26.069068 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-80ee0a61-a4be-47ed-9f4c-9e1db2e8246c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.045515 Oct 09 01:04:26.069653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 453/907] 10.222.0.33 () {62 vars in 1200 bytes} [Sun Oct 9 01:04:26 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 47 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 09 01:04:27.345207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-399553569", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d442b64f-563d-410a-8314-ec0e04e8ccd0"}], "security_groups": [{"name": "tempest-securitygroup--57864267"}], "key_name": "tempest-keypair-454303479", "user_data": "}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:27.447977 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] validate_networks() for [('d442b64f-563d-410a-8314-ec0e04e8ccd0', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:04:27.803485 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:04:27.803972 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:04:27.804460 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:04:27.804875 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:04:27.805290 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:04:27.813837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:27.814315 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:27.814758 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:27.815489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:27.815978 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:27.816997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:27.833689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for project 410212da4a794c07b30ed22cc5b7e760. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:04:27.837515 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Getting quotas for user 56b5d764f0044358ac594a2edd6f4c0f and project 410212da4a794c07b30ed22cc5b7e760. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:04:27.848235 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:04:27.850479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:04:27.898830 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a627a05c-608b-427f-b7ac-a091fde932b2 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 0.558563 Oct 09 01:04:27.905743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 455/908] 10.222.0.33 () {62 vars in 1188 bytes} [Sun Oct 9 01:04:27 2022] POST /compute/v2.1/servers => generated 400 bytes in 560 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 09 01:04:27.911808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-20c431be-1396-463c-a7f5-79d5a2aa691c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:27.912458 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-20c431be-1396-463c-a7f5-79d5a2aa691c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:27.970187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-20c431be-1396-463c-a7f5-79d5a2aa691c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:27.979474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-20c431be-1396-463c-a7f5-79d5a2aa691c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:27.980934 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-20c431be-1396-463c-a7f5-79d5a2aa691c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:27.982269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-20c431be-1396-463c-a7f5-79d5a2aa691c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1280 microversion: 2.20 time: 0.075006 Oct 09 01:04:27.982818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 454/909] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:27 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1280 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:29.007886 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:29.008561 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:29.014541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:29.015082 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:29.015527 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:29.016246 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:29.016705 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:29.017135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:29.082659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:29.090743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:29.092335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:29.093456 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a00095b0-0e64-4bb1-a221-c65d2f6db235 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1332 microversion: 2.20 time: 0.090658 Oct 09 01:04:29.094010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 456/910] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:29 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1332 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:30.119492 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:30.120143 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:30.130244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:30.130244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:30.130244 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:30.131289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:30.131695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:30.132129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:30.274343 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:30.284199 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:30.286256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:30.287368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-fcf1b37e-8f0b-4036-b245-e1789df68c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.172094 Oct 09 01:04:30.300010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 455/911] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:30 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:30.850408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-45cfcfe6-9a1e-4c0b-a860-d6c7c1a1191a req-1a8b63b8-bd42-40e7-bef0-0387609e08ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:30.861712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-45cfcfe6-9a1e-4c0b-a860-d6c7c1a1191a req-1a8b63b8-bd42-40e7-bef0-0387609e08ad service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:30.862283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-45cfcfe6-9a1e-4c0b-a860-d6c7c1a1191a req-1a8b63b8-bd42-40e7-bef0-0387609e08ad service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:30.862775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-45cfcfe6-9a1e-4c0b-a860-d6c7c1a1191a req-1a8b63b8-bd42-40e7-bef0-0387609e08ad service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:30.874445 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-45cfcfe6-9a1e-4c0b-a860-d6c7c1a1191a req-1a8b63b8-bd42-40e7-bef0-0387609e08ad service nova] Creating event network-changed:41d333d7-a33b-44cc-9e3c-2de2e0120491 for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 on hv-cinder-maste Oct 09 01:04:30.880604 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-45cfcfe6-9a1e-4c0b-a860-d6c7c1a1191a req-1a8b63b8-bd42-40e7-bef0-0387609e08ad service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039905 Oct 09 01:04:30.882357 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 457/912] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:31.313511 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:31.314292 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:31.321075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:31.321557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:31.321994 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:31.322685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:31.323143 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:31.323570 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:31.457193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:31.465197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:31.466653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:31.467768 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b4742eb6-679c-413a-b242-ae5d72a6cc79 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.158763 Oct 09 01:04:31.468321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 456/913] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:31 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:32.493400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:32.494089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:32.500210 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:32.500697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:32.501131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:32.501816 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:32.502268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:32.502689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:32.704170 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:32.712223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:32.713651 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:32.714746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-293d134a-36fe-4bc6-8519-fc9abb4ee490 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.226705 Oct 09 01:04:32.715300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 458/914] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:32 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:33.741070 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:33.741671 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:33.748016 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:33.748485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:33.748910 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:33.749585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:33.750029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:33.750448 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:33.875047 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:33.883622 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:33.885061 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:33.886147 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a8efb860-0a81-4ddc-b97b-022e7bb7c0b5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.149549 Oct 09 01:04:33.886688 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 457/915] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:33 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:34.912141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:34.912738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:34.919266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:34.919922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:34.920363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:34.921046 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:34.921498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:34.921915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:35.049606 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:35.057498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:35.058946 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:35.060067 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5c0c119e-0a02-4be0-9c90-3156288e27d6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.151870 Oct 09 01:04:35.060613 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 459/916] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:34 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:36.085185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:36.085784 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:36.088276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:36.094048 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:36.094539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:36.094977 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:36.095703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:36.096160 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:36.096587 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:36.226303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:36.234200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:36.235699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:36.236787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8acb5367-a8bd-422e-9a60-905981dcdea0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.155570 Oct 09 01:04:36.237338 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 458/917] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:36 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:36.846394 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-67a56dce-a046-4812-b6b6-96711c49ceb9 req-d212870c-3503-448c-9358-e990877885a5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "name": "network-vif-plugged", "status": "completed", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:36.857728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-67a56dce-a046-4812-b6b6-96711c49ceb9 req-d212870c-3503-448c-9358-e990877885a5 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:36.858212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-67a56dce-a046-4812-b6b6-96711c49ceb9 req-d212870c-3503-448c-9358-e990877885a5 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:36.858639 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-67a56dce-a046-4812-b6b6-96711c49ceb9 req-d212870c-3503-448c-9358-e990877885a5 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:36.869953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-67a56dce-a046-4812-b6b6-96711c49ceb9 req-d212870c-3503-448c-9358-e990877885a5 service nova] Creating event network-vif-plugged:41d333d7-a33b-44cc-9e3c-2de2e0120491 for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 on hv-cinder-maste Oct 09 01:04:36.874913 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-67a56dce-a046-4812-b6b6-96711c49ceb9 req-d212870c-3503-448c-9358-e990877885a5 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032237 Oct 09 01:04:36.875550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 460/918] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:04:37.262129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:37.262736 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:37.269734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:37.270222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:37.270659 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:37.271339 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:37.271824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:37.272246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:37.398094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:37.406412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:37.407876 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:37.408983 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-333d3514-c0fa-4e65-8d9e-4fecf2e8304c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.150732 Oct 09 01:04:37.409539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 459/919] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:37 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:38.431200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:38.432018 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:38.436472 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:38.437164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:38.444652 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:38.445135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:38.445600 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:38.446289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:38.446742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:38.447166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:38.573607 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:38.581649 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:38.583090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:38.584222 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-591002fe-ee58-49a1-9f73-336d83437fce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1404 microversion: 2.20 time: 0.153975 Oct 09 01:04:38.584769 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 461/920] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:38 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1404 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:39.608580 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:39.609182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:39.611109 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=6 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:39.617512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:39.618018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:39.618452 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:39.619134 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:39.619604 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:39.620034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:39.746536 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:39.754538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:39.756009 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:39.757147 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b46b0f2e-90ed-4ffc-b483-814dd080d361 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1596 microversion: 2.20 time: 0.152484 Oct 09 01:04:39.757699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 460/921] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:04:39 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1596 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:04:39.781816 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:39.782322 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:39.785234 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=58,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:39.790323 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:39.790804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:39.791241 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:39.791948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:39.792402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:39.792824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:39.872797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-50f30018-d148-4d21-a5d5-79086a777339 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.094831 Oct 09 01:04:39.873396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 462/922] 10.222.0.33 () {60 vars in 1316 bytes} [Sun Oct 9 01:04:39 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:04:40.780991 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-aff579e3-29a3-4753-a223-ee9fb441b935 req-ff97c8da-a8c1-41eb-a2de-63908d21af82 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:40.790668 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-aff579e3-29a3-4753-a223-ee9fb441b935 req-ff97c8da-a8c1-41eb-a2de-63908d21af82 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:40.791229 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-aff579e3-29a3-4753-a223-ee9fb441b935 req-ff97c8da-a8c1-41eb-a2de-63908d21af82 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:40.791802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-aff579e3-29a3-4753-a223-ee9fb441b935 req-ff97c8da-a8c1-41eb-a2de-63908d21af82 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:40.802997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-aff579e3-29a3-4753-a223-ee9fb441b935 req-ff97c8da-a8c1-41eb-a2de-63908d21af82 service nova] Creating event network-changed:41d333d7-a33b-44cc-9e3c-2de2e0120491 for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 on hv-cinder-maste Oct 09 01:04:40.808330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-aff579e3-29a3-4753-a223-ee9fb441b935 req-ff97c8da-a8c1-41eb-a2de-63908d21af82 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031078 Oct 09 01:04:40.808935 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 461/923] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:04:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:04:46.647291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:04:46.649557 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:46.655767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:46.656250 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:46.656686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:46.657368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:46.657819 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:46.658243 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:46.737144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ad1ce560-ff83-47ac-b13a-1f905a59b15a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/action" status: 202 len: 0 microversion: 2.1 time: 0.094046 Oct 09 01:04:46.741171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 463/924] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 01:04:46 2022] POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/action => generated 0 bytes in 96 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:04:46.747515 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:46.748835 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:46.757936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:46.758510 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:46.759051 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:46.759941 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:46.760360 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:46.760806 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:46.908274 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:46.917263 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-42e81225-4ee6-4054-84af-bf94a12f32a4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1751 microversion: 2.1 time: 0.174150 Oct 09 01:04:46.917826 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 462/925] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:46 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1751 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:04:47.942771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:47.943727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:47.950032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:47.950529 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:47.951006 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:47.951731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:47.952193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:47.952622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:48.106498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:48.114622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-960c29fa-badd-404e-ba70-89fcf79a456f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1751 microversion: 2.1 time: 0.176737 Oct 09 01:04:48.115182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 464/926] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:47 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1751 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:49.137734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:49.138342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:49.144849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:49.145325 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:49.145761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:49.146442 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:49.146892 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:49.147314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:49.289493 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:49.297762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5734cfd7-4d86-4c25-a574-3207eb1f4940 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.163935 Oct 09 01:04:49.298324 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 463/927] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:49 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:50.322203 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:50.322745 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:50.329228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:50.329702 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:50.330134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:50.330807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:50.331253 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:50.331699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:50.468708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:50.477327 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9abe5e8a-a9ea-45c3-a75f-1ac856a14e49 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.159120 Oct 09 01:04:50.477879 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 465/928] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:50 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:51.500353 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:51.500863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:51.507516 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:51.508024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:51.508461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:51.509146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:51.509597 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:51.510018 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:51.513969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=71,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:51.653438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:51.661987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-73c18ba1-03d0-48eb-b1a9-86ffeaf1f21a tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.165577 Oct 09 01:04:51.662542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 464/929] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:51 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:52.684632 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:52.685218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:52.692206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:52.692683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:52.693115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:52.693802 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:52.694251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:52.694675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:52.835687 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:52.843845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-093c51e0-3cb1-4a8c-8ea9-e5e2a637742f tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.163011 Oct 09 01:04:52.844407 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 466/930] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:52 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:53.867572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:53.868205 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:53.874839 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:53.875320 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:53.875934 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:53.876624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:53.877081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:53.877512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:54.020627 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:54.029158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b5c4b9b4-4ac3-4618-89a7-1ecd9226dbe5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.165525 Oct 09 01:04:54.029718 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 465/931] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:53 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:55.052688 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:55.053329 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:55.059649 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:55.060135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:55.060571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:55.061258 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:55.061710 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:55.062136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:55.064738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=76,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:04:55.202229 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:55.211086 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2aa82dd3-bed5-454d-a7bb-f180f6f628d5 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.162387 Oct 09 01:04:55.211671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 467/932] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:55 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:56.234488 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:56.235097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:56.242284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:56.242772 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:56.243213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:56.243966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:56.244427 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:56.244856 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:56.385077 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:56.393196 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ecf2750e-4260-49e8-9134-0c16eca8f4ac tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.162785 Oct 09 01:04:56.393766 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 466/933] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:56 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:57.416915 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:57.417539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:57.423895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:57.424370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:57.424805 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:57.425510 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:57.425958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:57.426377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:57.563981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:57.572751 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-575e5852-6eff-4df3-b2e3-c921056b6373 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.159704 Oct 09 01:04:57.573309 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 468/934] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:57 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:58.595573 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:58.596363 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:58.603067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:58.603538 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:58.604044 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:58.604733 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:58.605178 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:58.605599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:58.747438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:58.755455 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-015a1971-df44-4766-92e7-38d7d22e78a2 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.163678 Oct 09 01:04:58.756038 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 467/935] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:58 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:04:59.779409 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:04:59.780214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:04:59.786911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:59.787390 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:59.787850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:59.788530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:04:59.788976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:04:59.789395 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:04:59.933479 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:04:59.941599 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a3fd0400-3db9-4682-b105-0dfc601cd8a7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.166340 Oct 09 01:04:59.942338 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 469/936] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:04:59 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:00.965163 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:00.965763 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:00.972264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:00.972742 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:00.973177 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:00.973854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:00.974302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:00.974723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:01.118560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:01.126834 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-54f7976e-fa76-45f1-9ef7-f0249e950533 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.165484 Oct 09 01:05:01.127400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 468/937] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:00 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:02.152587 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:02.153524 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:02.160806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:02.161312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:02.161772 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:02.162462 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:02.162919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:02.163342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:02.302021 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:02.310097 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-aa548d06-8b80-45c0-ad7a-d07b3dc9de01 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.162521 Oct 09 01:05:02.310664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 470/938] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:02 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:05:03.333543 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:03.334088 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:03.340629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:03.341112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:03.341546 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:03.342234 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:03.342685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:03.343109 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:03.490952 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:03.499180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-37a9f5c5-1f4d-4e0c-9be7-05c63686ba3c tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1756 microversion: 2.1 time: 0.169528 Oct 09 01:05:03.499772 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 469/939] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:03 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1756 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:04.522582 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:04.523190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:04.529858 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:04.530336 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:04.530778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:04.531455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:04.531934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:04.532358 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:04.675069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:04.683141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f47da28b-03bd-4c73-9306-b7619c016d37 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1759 microversion: 2.1 time: 0.164980 Oct 09 01:05:04.683724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 471/940] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:04 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1759 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:16.622306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:16.622868 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:16.625294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=23 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:16.625947 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=35 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:16.631690 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:16.632172 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:16.632611 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:16.633298 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:16.633777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:16.634821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:16.657917 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-053cd1ea-e0eb-4f7f-9ca8-9b56591bda54 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/ips" status: 200 len: 152 microversion: 2.20 time: 0.040130 Oct 09 01:05:16.658467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 470/941] 10.222.0.33 () {60 vars in 1289 bytes} [Sun Oct 9 01:05:16 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/ips => generated 152 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:05:18.576979 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:18.577565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:18.580051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=19 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:18.580704 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=29 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:18.586365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:18.586851 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:18.588013 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:18.588655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:18.589093 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:18.589522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:18.612446 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lazy-loading 'flavor' on Instance uuid 406c4931-6ce2-45ec-bea3-b60d04e150c9 {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:05:18.795359 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-483eb794-3930-4a1a-babc-a617334d0227 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/action" status: 202 len: 0 microversion: 2.20 time: 0.222157 Oct 09 01:05:18.796271 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 472/942] 10.222.0.33 () {62 vars in 1319 bytes} [Sun Oct 9 01:05:18 2022] POST /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/action => generated 0 bytes in 224 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:05:18.804365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:18.804975 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:18.811549 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:18.812133 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:18.812512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:18.813206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:18.813657 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:18.814073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:18.944714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:18.953197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:18.954613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:18.955769 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-14fcb69b-ccd0-4be9-9b0d-12e2bef5a388 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.155239 Oct 09 01:05:18.956324 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 471/943] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:18 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:19.982963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:19.983675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:19.989872 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:19.990352 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:19.990789 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:19.991477 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:19.991952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:19.992381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:20.122591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:20.130552 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:20.132012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:20.133168 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-29ac4ccc-bd63-42c1-885a-0160765beb51 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.155551 Oct 09 01:05:20.133721 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 473/944] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:19 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:21.158594 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:21.159195 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:21.165631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:21.166105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:21.166540 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:21.167223 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:21.167705 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:21.168132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:21.299836 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:21.308568 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:21.310039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:21.311230 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a648c06a-7152-464c-a679-8c8310f1a31e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.156507 Oct 09 01:05:21.311834 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 472/945] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:21 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:22.336668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:22.337731 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:22.344060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:22.344539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:22.344977 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:22.345664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:22.346112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:22.346533 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:22.474513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:22.482995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:22.484476 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:22.485629 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2517374a-12f9-4509-94a0-ed50caef8e31 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.152961 Oct 09 01:05:22.486184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 474/946] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:22 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:23.510854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:23.511387 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:23.517852 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:23.518326 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:23.518762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:23.519445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:23.519920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:23.520344 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:23.648434 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:23.657100 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:23.658530 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:23.659967 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7a2412c0-0d0c-4473-9006-7f54efdc86a5 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.153080 Oct 09 01:05:23.660528 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 473/947] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:23 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:24.686771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:24.687377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:24.693719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:24.694200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:24.694634 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:24.695318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:24.695795 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:24.696217 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:24.820921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:24.829380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:24.830816 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:24.831992 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-da31fb22-6ec5-4a92-a8da-c9a4024fada3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.149159 Oct 09 01:05:24.832553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 475/948] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:24 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:25.856925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:25.857531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:25.864086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:25.864567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:25.865004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:25.865687 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:25.866141 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:25.866565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:25.993019 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:26.000986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:26.002448 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:26.003583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e168416c-33f3-42a2-8219-48b05aee0c6f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.150590 Oct 09 01:05:26.004157 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 474/949] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:25 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:27.030665 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:27.031365 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:27.037392 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:27.037884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:27.038325 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:27.039024 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:27.039481 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:27.039942 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:27.177444 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:27.186100 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:27.187617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:27.188839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-804919c9-8f95-4a2a-a3df-b912b8395203 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.163006 Oct 09 01:05:27.189393 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 476/950] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:27 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:28.215468 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:28.216104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:28.222342 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:28.222823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:28.223259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:28.223973 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:28.224421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:28.224840 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:28.357966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:28.365921 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:28.367352 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:28.369079 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-73ed10eb-1e17-4321-aa49-8b57ff895727 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.157921 Oct 09 01:05:28.369633 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 475/951] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:28 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:05:29.394739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:29.395289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:29.402522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:29.403001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:29.403434 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:29.404136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:29.404586 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:29.405007 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:29.531268 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:29.539200 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:29.540683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:29.541839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-79d30483-2629-4c85-a682-b8024e27cd77 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.151079 Oct 09 01:05:29.542391 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 477/952] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:29 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:30.566662 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:30.567264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:30.573658 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:30.574139 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:30.574583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:30.575271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:30.575755 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:30.576183 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:30.701045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:30.715087 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:30.716548 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:30.717715 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-56b40a8b-c464-4712-ad13-6c608b14d373 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.154919 Oct 09 01:05:30.718271 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 476/953] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:30 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:31.744903 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:31.745496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:31.751698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:31.752208 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:31.752644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:31.753327 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:31.753779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:31.754229 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:31.878351 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:31.886326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:31.887804 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:31.888996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-5dcd4fee-85ab-4d40-b333-1a2321fe6140 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.148554 Oct 09 01:05:31.889550 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 478/954] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:31 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:32.914599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:32.915192 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:32.921616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:32.922105 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:32.922543 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:32.923234 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:32.923716 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:32.924146 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:33.053006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:33.061625 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:33.063075 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:33.064265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d188c9ae-30c5-4158-a60a-b1c25839465b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.153587 Oct 09 01:05:33.064818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 477/955] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:32 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:34.090963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:34.091567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:34.097904 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:34.098386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:34.098826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:34.099509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:34.099983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:34.100408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:34.229419 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:34.237484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:34.238921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:34.240109 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-211c6d85-d5dc-43a8-900a-9ec65a7cd025 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.153822 Oct 09 01:05:34.240663 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 479/956] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:34 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:35.265414 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:35.266020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:35.272410 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:35.272881 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:35.273315 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:35.273990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:35.274955 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:35.275371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:35.400635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:35.408872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:35.410304 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:35.411437 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-dce54310-b8d4-4bbe-a37e-fb1b33a03f2b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1742 microversion: 2.20 time: 0.149953 Oct 09 01:05:35.412032 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 478/957] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:35 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1742 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:36.437827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:36.438433 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:36.444976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:36.445456 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:36.445926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:36.446659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:36.447103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:36.447522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:36.594188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:36.603713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:36.605567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:36.606318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-fe4e8970-f940-47b1-b3c1-4160ca6837b8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.172540 Oct 09 01:05:36.606865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 480/958] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:36 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:37.635829 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:37.636472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:37.643873 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:37.644599 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:37.645222 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:37.645923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:37.646378 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:37.646802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:37.795867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:37.806586 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:37.808060 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:37.809233 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-13174ad5-40e2-42af-99be-73803502335f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.179317 Oct 09 01:05:37.810048 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 479/959] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:37 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:38.135843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1915066342"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:38.137108 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:38.144400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:38.145164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:38.145619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:38.146507 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:38.147128 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:38.147555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:38.295792 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.network.security_group_api [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Removing security group dc056f2c-4cfe-4712-a0d3-62e45f5a9644 from port ebfdac77-d547-424a-bd77-91436ba7750c Oct 09 01:05:38.760280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-47484a31-7665-4b43-a0fc-15a8c9d510e0 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/action" status: 202 len: 0 microversion: 2.1 time: 0.629126 Oct 09 01:05:38.761569 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 481/960] 10.222.0.33 () {60 vars in 1278 bytes} [Sun Oct 9 01:05:38 2022] POST /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/action => generated 0 bytes in 631 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:05:38.773714 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-6817b448-fccf-47f9-acf5-556a4e5e4b7b req-6ec0d89b-55ab-40ce-92d1-217da85bf222 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:38.798685 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-6817b448-fccf-47f9-acf5-556a4e5e4b7b req-6ec0d89b-55ab-40ce-92d1-217da85bf222 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:38.799159 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-6817b448-fccf-47f9-acf5-556a4e5e4b7b req-6ec0d89b-55ab-40ce-92d1-217da85bf222 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:38.799583 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-6817b448-fccf-47f9-acf5-556a4e5e4b7b req-6ec0d89b-55ab-40ce-92d1-217da85bf222 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:38.813082 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-6817b448-fccf-47f9-acf5-556a4e5e4b7b req-6ec0d89b-55ab-40ce-92d1-217da85bf222 service nova] Creating event network-changed:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:05:38.827202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-6817b448-fccf-47f9-acf5-556a4e5e4b7b req-6ec0d89b-55ab-40ce-92d1-217da85bf222 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059855 Oct 09 01:05:38.837527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 480/961] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:38.840146 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:38.841194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:38.848651 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:38.849466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:38.850122 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:38.851149 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:38.852051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:38.852738 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:39.147665 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:39.165984 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:39.185918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:39.186825 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c1e931a1-e0c3-41ab-b387-bf44b16be5d8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.354224 Oct 09 01:05:39.193134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 482/962] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:38 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:39.387269 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-222ad49d-6017-462c-93bb-015f75925d4d req-7a55e125-63a4-47d3-9351-c95de87abe14 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "name": "network-vif-plugged", "status": "completed", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:39.409804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-222ad49d-6017-462c-93bb-015f75925d4d req-7a55e125-63a4-47d3-9351-c95de87abe14 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:39.413441 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-222ad49d-6017-462c-93bb-015f75925d4d req-7a55e125-63a4-47d3-9351-c95de87abe14 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:39.414121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-222ad49d-6017-462c-93bb-015f75925d4d req-7a55e125-63a4-47d3-9351-c95de87abe14 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:39.441827 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-222ad49d-6017-462c-93bb-015f75925d4d req-7a55e125-63a4-47d3-9351-c95de87abe14 service nova] Creating event network-vif-plugged:41d333d7-a33b-44cc-9e3c-2de2e0120491 for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 on hv-cinder-maste Oct 09 01:05:39.455522 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-222ad49d-6017-462c-93bb-015f75925d4d req-7a55e125-63a4-47d3-9351-c95de87abe14 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092753 Oct 09 01:05:39.457966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 481/963] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:40.222847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:40.224644 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:40.232399 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.233229 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.234308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.235219 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.235911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.236340 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.390934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:40.408098 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:40.409843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:40.411291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0c669e76-5df4-457c-b54c-8e0bb9779c0e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.193373 Oct 09 01:05:40.424839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 483/964] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:40 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:40.454374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:40.454955 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:40.462697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-f2f43dfd-0f3d-4e15-a139-a699f643228b req-1301ac72-76a7-4773-9e15-f99087b82946 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:40.466804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.467650 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.473422 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.474498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.475036 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.475708 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.487248 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-f2f43dfd-0f3d-4e15-a139-a699f643228b req-1301ac72-76a7-4773-9e15-f99087b82946 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.488115 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-f2f43dfd-0f3d-4e15-a139-a699f643228b req-1301ac72-76a7-4773-9e15-f99087b82946 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.496742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-f2f43dfd-0f3d-4e15-a139-a699f643228b req-1301ac72-76a7-4773-9e15-f99087b82946 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.509862 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-f2f43dfd-0f3d-4e15-a139-a699f643228b req-1301ac72-76a7-4773-9e15-f99087b82946 service nova] Creating event network-changed:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:05:40.515421 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-f2f43dfd-0f3d-4e15-a139-a699f643228b req-1301ac72-76a7-4773-9e15-f99087b82946 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056366 Oct 09 01:05:40.516630 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 484/965] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:40.606461 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lazy-loading 'services' on Instance uuid aa737c98-4038-4554-80d9-17b8cbd242c9 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:05:40.723838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b2c95d5d-f1bb-40b8-93c7-4b84b7aa6bae tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments/c0244eb3-4652-4f88-b2c3-e7478d390ae2" status: 202 len: 0 microversion: 2.1 time: 0.274464 Oct 09 01:05:40.724804 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 482/966] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:05:40 2022] DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments/c0244eb3-4652-4f88-b2c3-e7478d390ae2 => generated 0 bytes in 277 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:05:40.734259 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:40.736900 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:40.744084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.744837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.745447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.746375 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:40.746990 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:40.747578 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:40.822575 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-496e3890-dcfa-43e3-bd60-580078438b12 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092516 Oct 09 01:05:40.832110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 485/967] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:05:40 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments => generated 197 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:41.440068 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:41.441303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:41.455849 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:41.456489 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:41.457042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:41.457854 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:41.458550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:41.459080 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:41.721385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:41.738867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:41.743522 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:41.746779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9e9590ae-5241-4914-a24a-5ce9b293e784 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.312072 Oct 09 01:05:41.747541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 483/968] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:41 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:41.836140 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:41.840601 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:41.855001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:41.855504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:41.855979 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:41.856697 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:41.857162 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:41.859987 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:41.897087 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-05094d30-404a-4d26-89ce-37ca0a031cff tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.065274 Oct 09 01:05:41.898664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 486/969] 10.222.0.33 () {58 vars in 1302 bytes} [Sun Oct 9 01:05:41 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments => generated 25 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 09 01:05:41.913022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:41.914311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:41.923474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:41.924136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:41.924577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:41.925335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:41.925793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:41.926268 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:42.013667 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] HTTP exception thrown: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 is not attached to volume c0244eb3-4652-4f88-b2c3-e7478d390ae2 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Request method failure captured: Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments/c0244eb3-4652-4f88-b2c3-e7478d390ae2 HTTP/1.1 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh3i2qMxMvsJaZYUIW6eP3h202O0DaRxCpISXn8AQMcMftD5cg4QEi2PBXMP6v5xp_gTNE45HCCG13_FlJjDiHa-3HIqsX4yLIiHJNucb3Ubk-T9m5GvF4Qp12vAo21U7dPMTvjzy1I3yNXrLikHcxgrjVIQxt-sepIAFqo527wC23Y Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: f37de57a22a246f0935f85392898dd42 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-TestMinimumBasicScenario-747022156 Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:05:42.016314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f37de57a22a246f0935f85392898dd42"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f37de57a22a246f0935f85392898dd42"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f37de57a22a246f0935f85392898dd42"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f37de57a22a246f0935f85392898dd42"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-TestMinimumBasicScenario-747022156 Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: f37de57a22a246f0935f85392898dd42 Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-TestMinimumBasicScenario-747022156 Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-TestMinimumBasicScenario-747022156-project Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 3650b0b2b7014e0f8a7a5af9ce92374a Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-TestMinimumBasicScenario-747022156-project Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:05:42.017801 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 is not attached to volume c0244eb3-4652-4f88-b2c3-e7478d390ae2 Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'server_id': 'aa737c98-4038-4554-80d9-17b8cbd242c9', 'id': 'c0244eb3-4652-4f88-b2c3-e7478d390ae2'} Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id c0244eb3-4652-4f88-b2c3-e7478d390ae2. Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:05:42.018613 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 is not attached to volume c0244eb3-4652-4f88-b2c3-e7478d390ae2 Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 is not attached to volume c0244eb3-4652-4f88-b2c3-e7478d390ae2 Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:42.020386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Returning 404 to user: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 is not attached to volume c0244eb3-4652-4f88-b2c3-e7478d390ae2 {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:05:42.021346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a63a27f9-8c88-46e5-91ee-459d0cdee872 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments/c0244eb3-4652-4f88-b2c3-e7478d390ae2" status: 404 len: 154 microversion: 2.1 time: 0.111490 Oct 09 01:05:42.021346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 484/970] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:05:41 2022] DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9/os-volume_attachments/c0244eb3-4652-4f88-b2c3-e7478d390ae2 => generated 154 bytes in 113 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:05:42.776614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:42.777581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:42.784771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:42.785850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:42.786361 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:42.787163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:42.787827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:42.788318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:42.809655 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:43.001710 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:43.012193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:43.013771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:43.015231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e8999cb2-8950-4105-8400-340a1e9ad3c0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.244290 Oct 09 01:05:43.016034 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 487/971] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:42 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:43.214182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:43.215336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:43.222004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.222770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.223429 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.224226 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.224745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.225258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.260400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:05:43.347986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:43.395940 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-505b7d74-9c2e-4d05-8cbc-9f80ec67e64e tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 204 len: 0 microversion: 2.1 time: 0.185698 Oct 09 01:05:43.397551 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 485/972] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:05:43 2022] DELETE /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:05:43.408532 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:43.409211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:43.416141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.416627 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.417306 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.418166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.418650 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.419252 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.560808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:43.569752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-524fa699-47da-40d9-8114-21d84bdd1c43 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1554 microversion: 2.1 time: 0.166365 Oct 09 01:05:43.575664 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 488/973] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:43 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1554 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:43.809723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:43.810313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:43.820963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.821498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.821937 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.822654 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.823122 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.823537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.830759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-a0250336-f69f-4181-81ec-9b092f59ec29 req-72b3487d-576e-4805-ab4d-286322b30935 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e8b0e22-9d18-412f-97b9-bf5e750c9dcb", "tag": "07d4f0c5-1202-4554-b832-08bb967ca163"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:43.840682 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a0250336-f69f-4181-81ec-9b092f59ec29 req-72b3487d-576e-4805-ab4d-286322b30935 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.841470 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a0250336-f69f-4181-81ec-9b092f59ec29 req-72b3487d-576e-4805-ab4d-286322b30935 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.842091 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-a0250336-f69f-4181-81ec-9b092f59ec29 req-72b3487d-576e-4805-ab4d-286322b30935 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.855039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:05:43.856563 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-a0250336-f69f-4181-81ec-9b092f59ec29 req-72b3487d-576e-4805-ab4d-286322b30935 service nova] Creating event network-changed:07d4f0c5-1202-4554-b832-08bb967ca163 for instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb on hv-cinder-maste Oct 09 01:05:43.863312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-a0250336-f69f-4181-81ec-9b092f59ec29 req-72b3487d-576e-4805-ab4d-286322b30935 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036026 Oct 09 01:05:43.864652 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 489/974] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:43.972243 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cb9e50e5-2c9e-4690-9c3a-d1d85b816ff7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "DELETE /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 204 len: 0 microversion: 2.1 time: 0.166876 Oct 09 01:05:43.973174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 486/975] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:05:43 2022] DELETE /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:05:43.982670 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:43.983421 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:43.991570 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.992231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.993004 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:43.994907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:43.995402 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:43.995989 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:44.047050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:44.048176 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:44.064241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:44.065091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:44.065550 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:44.066266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:44.066724 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:44.067257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:44.204771 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:44.215840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-637dcde4-b7ed-4c6d-b48e-d7f231bedb5d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1578 microversion: 2.1 time: 0.237329 Oct 09 01:05:44.217156 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 490/976] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:43 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1578 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:44.304584 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:44.313431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:44.314931 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:44.316815 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-62165766-b21c-44e9-b2cc-6e96214dbdd4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.277550 Oct 09 01:05:44.331982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 487/977] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:44 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:44.595070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:44.596562 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:44.606328 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:44.607177 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:44.607985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:44.609090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:44.609090 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:44.609441 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:44.760733 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:44.773788 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-7e790999-d315-4021-9b56-d38b3be723c7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1554 microversion: 2.1 time: 0.182316 Oct 09 01:05:44.775645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 491/978] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:44 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1554 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:45.239695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:45.240406 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:45.247224 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:45.248045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:45.248693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:45.249631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:45.250153 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:45.250678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:45.344461 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:45.346571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:45.355070 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:45.355922 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:45.356592 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:45.357491 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:45.357950 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:45.358380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:45.458250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:45.470017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-d1b809a6-f273-4a5e-883a-ecc1734dbbc8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1578 microversion: 2.1 time: 0.234027 Oct 09 01:05:45.476389 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 488/979] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:45 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1578 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:45.512902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:45.521981 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:45.523102 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:45.524266 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1d6e71eb-64b9-44f1-ac1f-50125c770fcc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.185876 Oct 09 01:05:45.525369 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 492/980] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:45 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:05:45.799081 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:45.799641 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:45.806219 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:45.806712 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:45.807145 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:45.807887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:45.808341 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:45.808765 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:45.964593 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:45.973378 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7008907c-4355-47c3-9b85-f531d9746577 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1554 microversion: 2.1 time: 0.178496 Oct 09 01:05:45.974615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 489/981] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:45 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1554 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:46.508103 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:46.509245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:46.516128 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:46.516678 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:46.517111 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:46.517820 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:46.518270 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:46.518690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:46.550870 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:46.552863 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:46.560770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:46.561259 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:46.561691 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:46.562384 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:46.562832 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:46.563276 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:46.676937 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:46.687861 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-02d4c3f2-9ad9-4d06-b829-edf6b23140fd tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1578 microversion: 2.1 time: 0.183389 Oct 09 01:05:46.701066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 493/982] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:46 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1578 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:46.711093 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:46.721938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:46.723371 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:46.724612 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8738399e-0900-4224-97a9-02f0532d8be3 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1737 microversion: 2.20 time: 0.178042 Oct 09 01:05:46.725070 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 490/983] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:46 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:46.998698 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:46.999625 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:47.005775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.006263 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.006694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.007571 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.008284 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.008952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.255237 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3650b0b2b7014e0f8a7a5af9ce92374a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37de57a22a246f0935f85392898dd42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:47.287076 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-534e924f-e978-4b37-a95a-1ba13ba57db7 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 200 len: 1554 microversion: 2.1 time: 0.292422 Oct 09 01:05:47.289400 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 494/984] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:46 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 1554 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:47.321218 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-38657fdd-6587-452f-ac1a-4ec365fd6c29 req-1695dd46-3e86-47af-9685-117fc18161e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e8b0e22-9d18-412f-97b9-bf5e750c9dcb", "name": "network-vif-deleted", "tag": "07d4f0c5-1202-4554-b832-08bb967ca163"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:47.327187 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:47.334470 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-38657fdd-6587-452f-ac1a-4ec365fd6c29 req-1695dd46-3e86-47af-9685-117fc18161e4 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.335043 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-38657fdd-6587-452f-ac1a-4ec365fd6c29 req-1695dd46-3e86-47af-9685-117fc18161e4 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.335542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-38657fdd-6587-452f-ac1a-4ec365fd6c29 req-1695dd46-3e86-47af-9685-117fc18161e4 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.347618 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-38657fdd-6587-452f-ac1a-4ec365fd6c29 req-1695dd46-3e86-47af-9685-117fc18161e4 service nova] Creating event network-vif-deleted:07d4f0c5-1202-4554-b832-08bb967ca163 for instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb on hv-cinder-maste Oct 09 01:05:47.353280 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-38657fdd-6587-452f-ac1a-4ec365fd6c29 req-1695dd46-3e86-47af-9685-117fc18161e4 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035286 Oct 09 01:05:47.353865 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 491/985] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:47.519905 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-992e75e6-5093-4433-9255-b5c96c7e48ca req-1fe9af6a-16b6-4e84-b7d1-b9b87755b30e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa737c98-4038-4554-80d9-17b8cbd242c9", "name": "network-vif-deleted", "tag": "ebfdac77-d547-424a-bd77-91436ba7750c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:47.531797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-992e75e6-5093-4433-9255-b5c96c7e48ca req-1fe9af6a-16b6-4e84-b7d1-b9b87755b30e service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.532787 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-992e75e6-5093-4433-9255-b5c96c7e48ca req-1fe9af6a-16b6-4e84-b7d1-b9b87755b30e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.533390 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-992e75e6-5093-4433-9255-b5c96c7e48ca req-1fe9af6a-16b6-4e84-b7d1-b9b87755b30e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.545925 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-992e75e6-5093-4433-9255-b5c96c7e48ca req-1fe9af6a-16b6-4e84-b7d1-b9b87755b30e service nova] Creating event network-vif-deleted:ebfdac77-d547-424a-bd77-91436ba7750c for instance aa737c98-4038-4554-80d9-17b8cbd242c9 on hv-cinder-maste Oct 09 01:05:47.551495 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-992e75e6-5093-4433-9255-b5c96c7e48ca req-1fe9af6a-16b6-4e84-b7d1-b9b87755b30e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035458 Oct 09 01:05:47.552637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 495/986] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:47.715807 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:47.717875 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:47.727541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.728299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.728908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.729762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.730367 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.730943 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.750756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:47.751450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:47.761218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.763781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.764411 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.767237 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:47.767934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:47.770756 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:47.833961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:47.852785 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5c61a18d-2bf7-49e4-a3a5-8fe849209f2b tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1392 microversion: 2.1 time: 0.142976 Oct 09 01:05:47.859202 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 492/987] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:47 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1392 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:47.993840 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:48.002412 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:48.003893 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:48.005101 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1943690c-2fa5-4850-9f47-57e6a2f7edec tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1719 microversion: 2.20 time: 0.258634 Oct 09 01:05:48.005661 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 496/988] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:47 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1719 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:48.312503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:48.313110 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] [instance: aa737c98-4038-4554-80d9-17b8cbd242c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:48.319855 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:48.321083 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:48.321534 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:48.322311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:48.322942 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:48.323525 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:48.339264 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] HTTP exception thrown: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 could not be found. Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Request method failure captured: Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 HTTP/1.1 Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh3i2qMxMvsJaZYUIW6eP3h202O0DaRxCpISXn8AQMcMftD5cg4QEi2PBXMP6v5xp_gTNE45HCCG13_FlJjDiHa-3HIqsX4yLIiHJNucb3Ubk-T9m5GvF4Qp12vAo21U7dPMTvjzy1I3yNXrLikHcxgrjVIQxt-sepIAFqo527wC23Y Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: f37de57a22a246f0935f85392898dd42 Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-TestMinimumBasicScenario-747022156 Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:05:48.340972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_f37de57a22a246f0935f85392898dd42"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f37de57a22a246f0935f85392898dd42"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/f37de57a22a246f0935f85392898dd42"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/f37de57a22a246f0935f85392898dd42"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-TestMinimumBasicScenario-747022156 Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: f37de57a22a246f0935f85392898dd42 Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-TestMinimumBasicScenario-747022156 Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-TestMinimumBasicScenario-747022156-project Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 3650b0b2b7014e0f8a7a5af9ce92374a Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-TestMinimumBasicScenario-747022156-project Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 could not be found. Oct 09 01:05:48.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'aa737c98-4038-4554-80d9-17b8cbd242c9'} Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 could not be found. Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:05:48.343794 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 could not be found. Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 could not be found. Oct 09 01:05:48.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:48.347391 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Returning 404 to user: Instance aa737c98-4038-4554-80d9-17b8cbd242c9 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:05:48.348445 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9871d4b5-2347-4482-9698-748e71b32cd4 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9" status: 404 len: 111 microversion: 2.1 time: 0.040700 Oct 09 01:05:48.348996 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 493/989] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:48 2022] GET /compute/v2.1/servers/aa737c98-4038-4554-80d9-17b8cbd242c9 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:05:48.357852 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a766fa70-709e-4e28-b60f-7a96c59c4796 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] Calling method '.version_select at 0x7f7268a460d0>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:48.377825 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a766fa70-709e-4e28-b60f-7a96c59c4796 tempest-TestMinimumBasicScenario-747022156 tempest-TestMinimumBasicScenario-747022156-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-327703906" status: 202 len: 0 microversion: 2.1 time: 0.023726 Oct 09 01:05:48.378406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 497/990] 10.222.0.33 () {58 vars in 1269 bytes} [Sun Oct 9 01:05:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-327703906 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:05:48.877152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:48.877761 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:48.884982 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:48.885468 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:48.885908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:48.886596 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:48.887057 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:48.887482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:48.921423 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:48.922267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:48.922879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:48.923439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:05:49.031322 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:49.032228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:49.034801 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=54,nova_api:DELETE=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:49.041432 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:49.042041 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:49.042481 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:49.043231 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:49.043692 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:49.044119 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:49.182744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:49.187066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:49.192740 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-08d2c116-2df7-4f6b-89c0-766fa12371dc tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 200 len: 1397 microversion: 2.1 time: 0.319394 Oct 09 01:05:49.193409 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 494/991] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:48 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 1397 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:49.195286 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:49.196969 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:49.198134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6234f1d4-fad8-4527-b788-5757e4ea5623 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1719 microversion: 2.20 time: 0.171197 Oct 09 01:05:49.198676 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 498/992] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:49 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1719 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:05:50.215817 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:50.216367 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:50.223424 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:50.223963 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:50.224251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:50.224703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:50.225088 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:50.225207 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:50.225544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:50.225972 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:50.233106 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:50.233839 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:50.234971 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:50.235988 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:50.236541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:50.237111 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:50.244156 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] HTTP exception thrown: Instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb could not be found. Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Request method failure captured: Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb HTTP/1.1 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhyCzkbPt7AN3CTBkhz3r1POANnV9yaKeP2DZb2JPHpowTWCg7kviO4vrxDOli09Ug3OcMM-tmCxW5VH6AozJn5asWgto319ZeOwP7b0SF9yuk0cpFS3940Fk36BSylO6KrJAXuwZhEGiJ21v57PhWgfYWS3DrBgWsy-F7TQs1q5YO4 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:05:50.245802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-TestVolumeBootPattern-121914356 Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 04bc22d558c2466ba43a456d08bdd6bd Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb could not be found. Oct 09 01:05:50.247179 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '3e8b0e22-9d18-412f-97b9-bf5e750c9dcb'} Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb could not be found. Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:05:50.248368 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb could not be found. Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb could not be found. Oct 09 01:05:50.249565 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:05:50.250638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Returning 404 to user: Instance 3e8b0e22-9d18-412f-97b9-bf5e750c9dcb could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:05:50.250638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2e9c2f5f-46ed-4ecb-9ed1-960b51851e2a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb" status: 404 len: 111 microversion: 2.1 time: 0.035941 Oct 09 01:05:50.250638 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 495/993] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:50 2022] GET /compute/v2.1/servers/3e8b0e22-9d18-412f-97b9-bf5e750c9dcb => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:05:50.400724 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:50.410739 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:50.412251 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:50.413443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f915c895-24c3-49e8-83b4-7e10fc2c1f44 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1719 microversion: 2.20 time: 0.193734 Oct 09 01:05:50.414157 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 499/994] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:50 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1719 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:51.242042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-2e1e5a20-4873-4d3a-ae75-1241decfa6f5 req-0924036b-9ae4-4b61-b8d1-133aaa3d9fac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "name": "network-vif-unplugged", "status": "completed", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:51.271402 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-2e1e5a20-4873-4d3a-ae75-1241decfa6f5 req-0924036b-9ae4-4b61-b8d1-133aaa3d9fac service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:51.271918 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-2e1e5a20-4873-4d3a-ae75-1241decfa6f5 req-0924036b-9ae4-4b61-b8d1-133aaa3d9fac service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:51.272359 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-2e1e5a20-4873-4d3a-ae75-1241decfa6f5 req-0924036b-9ae4-4b61-b8d1-133aaa3d9fac service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:51.285814 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.compute.server_external_events [req-2e1e5a20-4873-4d3a-ae75-1241decfa6f5 req-0924036b-9ae4-4b61-b8d1-133aaa3d9fac service nova] Unable to find a host for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9. Dropping event network-vif-unplugged {{(pid=120632) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 09 01:05:51.286687 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-2e1e5a20-4873-4d3a-ae75-1241decfa6f5 req-0924036b-9ae4-4b61-b8d1-133aaa3d9fac service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.056935 Oct 09 01:05:51.287539 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 496/995] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:51 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 58 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 09 01:05:51.440546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:51.441560 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:51.448484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:51.449088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:51.450324 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:51.458312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:51.458899 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:51.459417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:51.654681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:51.665641 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:51.667182 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:51.668495 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-6917e777-7432-4922-8701-c9a728a0e3e6 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1662 microversion: 2.20 time: 0.232335 Oct 09 01:05:51.669275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 500/996] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:51 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1662 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:51.693194 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0399b8c1-d70e-48b0-967e-4fb4fbde0261"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:51.695497 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:51.701582 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:51.702231 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:51.702837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:51.703729 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:51.704333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:51.704915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:52.007197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c59e1969-cd69-47e9-a92f-bd1686525a37 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.317805 Oct 09 01:05:52.024720 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 497/997] 10.222.0.33 () {62 vars in 1364 bytes} [Sun Oct 9 01:05:51 2022] POST /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-volume_attachments => generated 188 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 09 01:05:52.174823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:52.175321 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:52.181884 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:52.182841 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:52.183782 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:52.184551 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:52.185014 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:52.185442 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:52.457159 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-bacb508c-0d6c-4dae-96c0-18acc6c1c937 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-volume_attachments/0399b8c1-d70e-48b0-967e-4fb4fbde0261" status: 202 len: 0 microversion: 2.20 time: 0.286561 Oct 09 01:05:52.458215 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 501/998] 10.222.0.33 () {60 vars in 1457 bytes} [Sun Oct 9 01:05:52 2022] DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-volume_attachments/0399b8c1-d70e-48b0-967e-4fb4fbde0261 => generated 0 bytes in 288 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:05:52.522446 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:52.523188 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:52.530925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:52.531758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:52.532331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:52.533150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:52.533806 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:52.534290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:52.606321 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120632) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 09 01:05:52.680196 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a83bbd32-330e-402c-9cef-2fdcdf9862da tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "POST /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/action" status: 202 len: 0 microversion: 2.20 time: 0.161709 Oct 09 01:05:52.682029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 498/999] 10.222.0.33 () {62 vars in 1319 bytes} [Sun Oct 9 01:05:52 2022] POST /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/action => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:05:52.690272 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:52.691415 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:52.700029 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:52.700844 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:52.701619 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:52.702556 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:52.703074 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:52.703746 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:53.060930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:53.072962 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:53.075201 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:53.076819 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9617d946-0070-45aa-af65-b63b9ea86e1b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1670 microversion: 2.20 time: 0.390802 Oct 09 01:05:53.078279 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 502/1000] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:52 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1670 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:05:53.742553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-1c2e6e03-c70a-42af-85b4-97c0f48a8d4f req-39908641-e6a0-46a8-a5d3-d3304cebe85a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "edbfc046-964c-46ac-9d30-5198c846e9b5", "tag": "92161e7f-b2e3-4be5-9ff8-073cd2234c8e"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:53.745148 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:53.745619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:53.759572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:53.760091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:53.760803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:53.761236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:53.761687 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:53.762107 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:53.770775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1c2e6e03-c70a-42af-85b4-97c0f48a8d4f req-39908641-e6a0-46a8-a5d3-d3304cebe85a service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:53.771543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1c2e6e03-c70a-42af-85b4-97c0f48a8d4f req-39908641-e6a0-46a8-a5d3-d3304cebe85a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:53.772258 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-1c2e6e03-c70a-42af-85b4-97c0f48a8d4f req-39908641-e6a0-46a8-a5d3-d3304cebe85a service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:53.787579 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-1c2e6e03-c70a-42af-85b4-97c0f48a8d4f req-39908641-e6a0-46a8-a5d3-d3304cebe85a service nova] Creating event network-changed:92161e7f-b2e3-4be5-9ff8-073cd2234c8e for instance edbfc046-964c-46ac-9d30-5198c846e9b5 on hv-cinder-maste Oct 09 01:05:53.793614 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=6,nova_cell1:INSERT=5 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:53.796188 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-1c2e6e03-c70a-42af-85b4-97c0f48a8d4f req-39908641-e6a0-46a8-a5d3-d3304cebe85a service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057296 Oct 09 01:05:53.797233 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 503/1001] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:53.800788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:05:53.895385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-968adcb2-19fe-48df-ad09-459aed2755ab tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "DELETE /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 204 len: 0 microversion: 2.1 time: 0.161300 Oct 09 01:05:53.896323 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 499/1002] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:05:53 2022] DELETE /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:05:53.905625 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:53.906447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:53.916075 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:53.916704 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:53.917261 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:53.918180 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:53.918729 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:53.919411 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:54.106990 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:54.107777 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:54.114369 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:54.115030 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:54.115656 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:54.116556 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:54.117015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:54.117440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:54.117614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:54.120028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:05:54.129949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ef63e33a-eeef-4045-9a9e-9e95871dbe16 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1577 microversion: 2.1 time: 0.228645 Oct 09 01:05:54.131377 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 504/1003] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:53 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1577 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:54.444326 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:54.454494 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:54.455992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:54.457501 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cd308f79-0c8e-4b03-9c96-7974b7a20ce8 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.356163 Oct 09 01:05:54.458531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 500/1004] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:54 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:55.158225 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:55.158947 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:55.166806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:55.167317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:55.172539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:55.173267 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:55.173742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:55.174170 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:55.484754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:55.486043 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:55.494741 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:55.504030 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0330b873-3fdd-4d50-9c73-a951b8ff3be5 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1577 microversion: 2.1 time: 0.354291 Oct 09 01:05:55.505086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:55.505870 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 505/1005] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:55 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1577 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:55.506571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:55.507126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:55.508086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:55.509314 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:55.509978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:55.843505 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:55.869796 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:55.872091 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:55.874190 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cb7da1bb-0040-4e01-89c4-c2567b74b278 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.392749 Oct 09 01:05:55.892748 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 501/1006] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:55 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:56.531986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:56.533449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:56.546413 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:56.546902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:56.548141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:56.560907 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:56.563401 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:56.563883 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:56.660842 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:56.669493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-3be71a1c-2c4b-43fc-a2fb-d56954309728 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1391 microversion: 2.1 time: 0.142876 Oct 09 01:05:56.670068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 506/1007] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:56 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1391 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:56.700641 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-7eba9d56-977c-4134-8c16-4464984f301d req-09e3320e-be09-4ae8-ba09-790f640b2eaa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "edbfc046-964c-46ac-9d30-5198c846e9b5", "name": "network-vif-deleted", "tag": "92161e7f-b2e3-4be5-9ff8-073cd2234c8e"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:05:56.710397 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-7eba9d56-977c-4134-8c16-4464984f301d req-09e3320e-be09-4ae8-ba09-790f640b2eaa service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:56.710890 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-7eba9d56-977c-4134-8c16-4464984f301d req-09e3320e-be09-4ae8-ba09-790f640b2eaa service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:56.711306 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-7eba9d56-977c-4134-8c16-4464984f301d req-09e3320e-be09-4ae8-ba09-790f640b2eaa service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:56.722258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-7eba9d56-977c-4134-8c16-4464984f301d req-09e3320e-be09-4ae8-ba09-790f640b2eaa service nova] Creating event network-vif-deleted:92161e7f-b2e3-4be5-9ff8-073cd2234c8e for instance edbfc046-964c-46ac-9d30-5198c846e9b5 on hv-cinder-maste Oct 09 01:05:56.727902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-7eba9d56-977c-4134-8c16-4464984f301d req-09e3320e-be09-4ae8-ba09-790f640b2eaa service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030126 Oct 09 01:05:56.728997 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 502/1008] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:05:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:05:56.901543 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:56.902072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:56.908344 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:56.911480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:56.911480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:56.911480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:56.911480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:56.912194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:57.089209 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:57.112135 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:57.114335 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:57.115754 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-012f1496-aea2-4129-9d52-061bd58c4ec9 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.218590 Oct 09 01:05:57.116743 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 507/1009] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:56 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:57.693830 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:57.694875 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:57.702215 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:57.703464 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:57.704817 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:57.706197 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:57.706874 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:57.707549 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:57.740162 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:57.740992 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:57.741624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:57.742235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:05:57.793284 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:57.824117 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8539e53d-d72a-4f9e-8b99-711c25973c2d tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.133020 Oct 09 01:05:57.824512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 503/1010] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:57 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:05:58.144581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:58.146345 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:58.154824 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:58.155867 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:58.156736 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:58.157530 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:58.158161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:58.158983 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:58.355342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:58.364374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:58.365827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:58.367001 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4d0a7311-248b-4d46-aca4-6e389942052c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.226584 Oct 09 01:05:58.367570 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 508/1011] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:58 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:58.847778 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:58.849065 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:58.859210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:58.859744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:58.860193 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:58.860908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:58.861364 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:58.861798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:58.895901 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:58.896397 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:58.896831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:58.897281 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:05:58.943652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:58.952575 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-52a0ed5d-2a18-41cf-9446-51672a361c3c tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.108652 Oct 09 01:05:58.953144 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 504/1012] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:58 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:05:59.392498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:59.393125 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:59.399837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:59.400622 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:59.401312 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:59.402300 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:59.403218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:59.403882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:59.547395 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:59.557130 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:59.558706 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:05:59.560026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0d0283a2-042b-439f-b5ff-2daa48a7823f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.171460 Oct 09 01:05:59.560681 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 509/1013] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:05:59 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:05:59.975965 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:05:59.978475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:05:59.987317 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:59.987923 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:59.988440 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:05:59.989250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:05:59.989779 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:05:59.990333 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:00.024957 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:00.025557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:00.026064 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:00.026745 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:06:00.069961 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:00.083635 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-13e600f4-43ab-487e-9658-d957b8e768ea tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.111368 Oct 09 01:06:00.084073 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 505/1014] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:05:59 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:00.586296 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:00.587573 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:00.593853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:00.594341 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:00.594779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:00.595475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:00.595952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:00.596378 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:00.736815 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:00.746226 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:00.747799 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:00.749317 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a460ef83-664d-4422-9c83-9de8fc0d3b71 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.166876 Oct 09 01:06:00.750073 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 510/1015] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:00 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:01.107160 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:01.107823 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:01.115390 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:01.115908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:01.116345 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:01.117042 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:01.117494 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:01.118482 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:01.151920 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:01.152409 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:01.153313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:01.153760 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:06:01.199365 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:01.207266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-f881e353-af77-4b29-a2b0-38eb66914ec8 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.104780 Oct 09 01:06:01.208046 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 506/1016] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:01 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:01.774298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:01.775052 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:01.781790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:01.782281 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:01.782718 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:01.783408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:01.783892 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:01.784316 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:01.934513 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:01.942896 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:01.944372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:01.945541 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-783a03a0-b3be-4729-8de2-cd0fc8c61d5f tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.175195 Oct 09 01:06:01.955856 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 511/1017] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:01 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:02.231838 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:02.232648 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:02.239968 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:02.240549 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:02.241099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:02.241938 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:02.242472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:02.243029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:02.276457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:02.277060 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:02.277590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:02.278318 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:06:02.322375 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:02.331472 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e272e6e4-9a9b-4269-bb9d-53ed0ca52b75 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.105285 Oct 09 01:06:02.334754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 507/1018] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:02 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:02.970536 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:02.972154 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:02.980131 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:02.980772 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:02.981332 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:02.982296 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:02.982853 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:02.983439 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:03.127447 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:03.137364 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:03.138821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:03.140193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ab9c9969-3168-4c31-9e2d-5e36f0e62239 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.173556 Oct 09 01:06:03.140758 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 512/1019] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:02 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:03.355498 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:03.356211 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:03.363388 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:03.365241 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:03.365955 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:03.366831 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:03.367457 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:03.368703 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:03.402099 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:03.402591 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:03.403282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:03.403754 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:06:03.450068 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:03.458316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8c2ad774-ead8-45a8-baa9-a81f31dc5c44 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.107217 Oct 09 01:06:03.459095 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 508/1020] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:03 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:04.168189 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:04.168913 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:04.176372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:04.176968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:04.177486 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:04.178753 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:04.178963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:04.179396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:04.338350 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:04.347759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:04.349194 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:04.350356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4f2c15bd-0210-4844-97c7-0953d016ce9d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.186672 Oct 09 01:06:04.350921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 513/1021] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:04 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:04.482557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:04.484267 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:04.493624 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:04.494203 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:04.494716 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:04.495487 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:04.496024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:04.496513 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:04.551235 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:04.551945 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:04.553180 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:04.553652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:06:04.598966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:04.607475 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-9f7e97f4-2d85-41e3-a572-db84bd3bf031 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.129849 Oct 09 01:06:04.608335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 509/1022] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:04 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:05.376370 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:05.376997 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:05.383888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:05.384381 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:05.384818 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:05.385499 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:05.385949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:05.386372 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:05.512779 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:05.520911 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:05.522326 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:05.523468 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e82651ff-13ca-41f1-b69d-9414496b6dce tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.151030 Oct 09 01:06:05.524053 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 514/1023] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:05 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:05.629821 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:05.630425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:05.636987 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:05.637469 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:05.637904 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:05.638587 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:05.639039 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:05.639465 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:05.669564 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:05.670033 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:05.670463 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:05.670902 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lazy-loading 'fault' on Instance uuid edbfc046-964c-46ac-9d30-5198c846e9b5 {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:06:05.713045 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04bc22d558c2466ba43a456d08bdd6bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28bc82345ea24b929b66fbcded2b97b7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:05.720576 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7854bb1e-4619-47eb-b5af-91e5a5b40af7 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 200 len: 1230 microversion: 2.1 time: 0.094649 Oct 09 01:06:05.721136 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 510/1024] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:05 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 1230 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:06.549596 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:06.550212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:06.556546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:06.557031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:06.557466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:06.558159 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:06.558614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:06.559034 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:06.742185 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:06.743319 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: edbfc046-964c-46ac-9d30-5198c846e9b5] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:06.749818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:06.750299 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:06.750737 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:06.751425 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:06.751906 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:06.752331 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:06.765555 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:06.766412 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] HTTP exception thrown: Instance edbfc046-964c-46ac-9d30-5198c846e9b5 could not be found. Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Request method failure captured: Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 HTTP/1.1 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhyCzkbPt7AN3CTBkhz3r1POANnV9yaKeP2DZb2JPHpowTWCg7kviO4vrxDOli09Ug3OcMM-tmCxW5VH6AozJn5asWgto319ZeOwP7b0SF9yuk0cpFS3940Fk36BSylO6KrJAXuwZhEGiJ21v57PhWgfYWS3DrBgWsy-F7TQs1q5YO4 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:06:06.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 04bc22d558c2466ba43a456d08bdd6bd Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance edbfc046-964c-46ac-9d30-5198c846e9b5 could not be found. Oct 09 01:06:06.769681 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': 'edbfc046-964c-46ac-9d30-5198c846e9b5'} Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance edbfc046-964c-46ac-9d30-5198c846e9b5 could not be found. Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:06.770503 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance edbfc046-964c-46ac-9d30-5198c846e9b5 could not be found. Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance edbfc046-964c-46ac-9d30-5198c846e9b5 could not be found. Oct 09 01:06:06.771872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:06.773108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Returning 404 to user: Instance edbfc046-964c-46ac-9d30-5198c846e9b5 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:06.773108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b1148a54-9972-4f29-a14c-d509d9338cfe tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5" status: 404 len: 111 microversion: 2.1 time: 0.031290 Oct 09 01:06:06.774531 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 511/1025] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:06 2022] GET /compute/v2.1/servers/edbfc046-964c-46ac-9d30-5198c846e9b5 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 09 01:06:06.774902 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:06.775519 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:06.776755 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-a7a55880-1c80-4c75-9dc3-c7f34c08a6fb tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.231110 Oct 09 01:06:06.777390 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 515/1026] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:06 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:07.230362 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:07.230871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:07.237469 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:07.237966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:07.238421 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:07.239132 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:07.239709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:07.240069 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:07.257936 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] HTTP exception thrown: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Request method failure captured: Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: DELETE /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 HTTP/1.1 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQhyCzkbPt7AN3CTBkhz3r1POANnV9yaKeP2DZb2JPHpowTWCg7kviO4vrxDOli09Ug3OcMM-tmCxW5VH6AozJn5asWgto319ZeOwP7b0SF9yuk0cpFS3940Fk36BSylO6KrJAXuwZhEGiJ21v57PhWgfYWS3DrBgWsy-F7TQs1q5YO4 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:06:07.259553 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 04bc22d558c2466ba43a456d08bdd6bd Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.261000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '841a7957-ee73-4765-a850-406b1e720612'} Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:07.261762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.262986 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.265771 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Returning 404 to user: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:07.266713 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c0a03d1b-1dfe-43e2-9dc7-40c8503e182a tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "DELETE /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 404 len: 111 microversion: 2.1 time: 0.040785 Oct 09 01:06:07.270676 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 512/1027] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:06:07 2022] DELETE /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:06:07.276374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:07.276994 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] [instance: 841a7957-ee73-4765-a850-406b1e720612] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:07.284526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:07.285040 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:07.285487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:07.286193 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:07.286647 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:07.287072 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:07.303514 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] HTTP exception thrown: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Request method failure captured: Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 HTTP/1.1 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQhyCzkbPt7AN3CTBkhz3r1POANnV9yaKeP2DZb2JPHpowTWCg7kviO4vrxDOli09Ug3OcMM-tmCxW5VH6AozJn5asWgto319ZeOwP7b0SF9yuk0cpFS3940Fk36BSylO6KrJAXuwZhEGiJ21v57PhWgfYWS3DrBgWsy-F7TQs1q5YO4 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:06:07.306083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/28bc82345ea24b929b66fbcded2b97b7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 28bc82345ea24b929b66fbcded2b97b7 Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-TestVolumeBootPattern-121914356 Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 04bc22d558c2466ba43a456d08bdd6bd Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-TestVolumeBootPattern-121914356-project Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '841a7957-ee73-4765-a850-406b1e720612'} Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:07.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. Oct 09 01:06:07.310144 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:07.311206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Returning 404 to user: Instance 841a7957-ee73-4765-a850-406b1e720612 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:07.311206 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-64dc4712-1341-40c3-a4d9-d4fc44598351 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612" status: 404 len: 111 microversion: 2.1 time: 0.035632 Oct 09 01:06:07.312176 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 516/1028] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:06:07 2022] GET /compute/v2.1/servers/841a7957-ee73-4765-a850-406b1e720612 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:06:07.802337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:07.803079 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:07.809636 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:07.810121 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:07.810560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:07.811256 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:07.811743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:07.812174 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:07.942255 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:07.946332 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:06:07.951786 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:07.953282 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:07.954476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-527b5c31-c343-4139-a6fe-2f86139baa5c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.156106 Oct 09 01:06:07.955031 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 513/1029] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:07 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:08.614023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-133a3f2e-0d0a-48a5-adf9-4faf32f28c04 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] Calling method '.version_select at 0x7f7268b3e700>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:08.638776 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-133a3f2e-0d0a-48a5-adf9-4faf32f28c04 tempest-TestVolumeBootPattern-121914356 tempest-TestVolumeBootPattern-121914356-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1094622065" status: 202 len: 0 microversion: 2.1 time: 0.028730 Oct 09 01:06:08.639376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 517/1030] 10.222.0.33 () {58 vars in 1263 bytes} [Sun Oct 9 01:06:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1094622065 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:06:08.980960 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:08.981677 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:08.989370 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:08.990113 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:08.990532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:08.991186 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:08.991669 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:08.992094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:09.131346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:09.139460 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:09.141006 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:09.142166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6645b264-42d2-4ec3-815f-481c4bdb536b tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.165683 Oct 09 01:06:09.142721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 514/1031] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:08 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:09.319918 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-9d9e4cba-6513-4f41-80ca-015baad809d4 req-f7d0819c-cf13-47d2-ae5a-2899a37832e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "name": "network-vif-plugged", "status": "completed", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:06:09.334136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-9d9e4cba-6513-4f41-80ca-015baad809d4 req-f7d0819c-cf13-47d2-ae5a-2899a37832e6 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:09.334949 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-9d9e4cba-6513-4f41-80ca-015baad809d4 req-f7d0819c-cf13-47d2-ae5a-2899a37832e6 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:09.335450 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-9d9e4cba-6513-4f41-80ca-015baad809d4 req-f7d0819c-cf13-47d2-ae5a-2899a37832e6 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:09.348778 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-9d9e4cba-6513-4f41-80ca-015baad809d4 req-f7d0819c-cf13-47d2-ae5a-2899a37832e6 service nova] Creating event network-vif-plugged:41d333d7-a33b-44cc-9e3c-2de2e0120491 for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 on hv-cinder-maste Oct 09 01:06:09.354823 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-9d9e4cba-6513-4f41-80ca-015baad809d4 req-f7d0819c-cf13-47d2-ae5a-2899a37832e6 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039173 Oct 09 01:06:09.355995 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 518/1032] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:06:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:06:10.169864 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:10.170467 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:10.177067 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:10.177544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:10.177971 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:10.178663 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:10.179112 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:10.179527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:10.313932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:10.322165 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:10.323632 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:10.324793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b6a8f920-3204-41d7-b4bc-ad78a585a01c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.158821 Oct 09 01:06:10.325358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 515/1033] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:10 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:11.351077 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:11.351796 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:11.379612 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:11.380488 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:11.388645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:11.389986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:11.393891 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:11.396807 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:11.647689 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:11.656861 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:11.658282 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:11.659443 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-df9a3662-5fbf-4fdf-af22-4c88cbe17c03 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1728 microversion: 2.20 time: 0.312677 Oct 09 01:06:11.660031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 519/1034] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:11 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1728 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:12.686266 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:12.686857 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:12.693228 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:12.693702 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:12.694133 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:12.694819 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:12.695270 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:12.696301 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:13.114150 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:13.122892 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:13.124463 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:13.125619 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-39593f62-3a39-424b-b789-0236422dc855 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1715 microversion: 2.20 time: 0.443290 Oct 09 01:06:13.126173 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 516/1035] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:12 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1715 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:43.991985 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:43.994171 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:43.996382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=44,nova_api:DELETE=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:06:43.997040 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=77 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:06:44.003002 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:44.003487 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:44.003952 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:44.004645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:44.005136 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:44.005566 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:44.035376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2417ca91-ef2a-4647-84e1-f68ad3262ae0 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.047572 Oct 09 01:06:44.036044 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 520/1036] 10.222.0.33 () {60 vars in 1343 bytes} [Sun Oct 9 01:06:43 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:06:44.099024 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:44.099524 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:44.101652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=78,nova_api:UPDATE=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:06:44.102536 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=18 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:06:44.108329 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:44.108816 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:44.109258 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:44.109953 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:44.110411 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:44.110845 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:44.136577 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:06:44.209000 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-124a23f2-4208-43ae-a1d2-c5f95543eb88 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 204 len: 0 microversion: 2.20 time: 0.113700 Oct 09 01:06:44.209649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 517/1037] 10.222.0.33 () {60 vars in 1280 bytes} [Sun Oct 9 01:06:44 2022] DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 09 01:06:44.217767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:44.218373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:44.224289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:44.224770 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:44.225204 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:44.225888 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:44.226337 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:44.226790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:44.356767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:44.364727 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:44.366166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:44.367831 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-dfd85265-e65c-4721-bc2d-ddcde74cbd0a tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.153476 Oct 09 01:06:44.368390 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 521/1038] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:44 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:45.393930 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:45.394649 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:45.400935 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:45.401407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:45.401837 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:45.402512 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:45.402959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:45.403374 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:45.536885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:45.545366 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:45.546787 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:45.547949 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-131ef4e9-4d91-4eda-8776-bf2fb76834e1 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1721 microversion: 2.20 time: 0.158379 Oct 09 01:06:45.548504 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 518/1039] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:45 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 09 01:06:46.574728 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:46.575614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:46.583429 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:46.583944 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:46.584380 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:46.585750 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:46.586221 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:46.586832 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:46.663882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:46.672668 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:46.674546 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56b5d764f0044358ac594a2edd6f4c0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '410212da4a794c07b30ed22cc5b7e760', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:06:46.675991 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-e3125a9e-d624-4103-9d92-66ea97c0c93c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 200 len: 1655 microversion: 2.20 time: 0.105793 Oct 09 01:06:46.676766 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 522/1040] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:46 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 1655 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 09 01:06:46.761124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-b0f02eaa-25f6-4eb3-848a-32260b5ff121 req-04715b52-8f47-4b6a-bf02-3dd53b48be71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "406c4931-6ce2-45ec-bea3-b60d04e150c9", "name": "network-vif-deleted", "tag": "41d333d7-a33b-44cc-9e3c-2de2e0120491"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:06:46.771138 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b0f02eaa-25f6-4eb3-848a-32260b5ff121 req-04715b52-8f47-4b6a-bf02-3dd53b48be71 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:46.771631 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b0f02eaa-25f6-4eb3-848a-32260b5ff121 req-04715b52-8f47-4b6a-bf02-3dd53b48be71 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:46.772261 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-b0f02eaa-25f6-4eb3-848a-32260b5ff121 req-04715b52-8f47-4b6a-bf02-3dd53b48be71 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:46.791428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-b0f02eaa-25f6-4eb3-848a-32260b5ff121 req-04715b52-8f47-4b6a-bf02-3dd53b48be71 service nova] Creating event network-vif-deleted:41d333d7-a33b-44cc-9e3c-2de2e0120491 for instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 on hv-cinder-maste Oct 09 01:06:46.797595 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-b0f02eaa-25f6-4eb3-848a-32260b5ff121 req-04715b52-8f47-4b6a-bf02-3dd53b48be71 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.208097 Oct 09 01:06:46.799020 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 519/1041] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:06:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 09 01:06:47.704636 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:47.705262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:47.711572 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:47.712085 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:47.712521 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:47.713223 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:47.713671 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:47.714089 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:47.727653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] HTTP exception thrown: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Request method failure captured: Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 HTTP/1.1 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh1zN0RFejgIlRejGebeuKod3w-XJOgFtBs1rs2AyaEyhIOmlH5SPOmzYiX14QlhxrykcEOZPGPR3baTn6hCjNBhJl6gEE4NxbzyP8nsobszHsmCZ9Ixv6cq69LQ6LeEoSfpt5SjWL79E30Tz4Y9e8xRrsDjNijB2ku6Nq_M4qVItyk Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Openstack-Nova-Api-Version: 2.20 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:06:47.729275 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 56b5d764f0044358ac594a2edd6f4c0f Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:47.731049 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '406c4931-6ce2-45ec-bea3-b60d04e150c9'} Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:47.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:47.733759 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:47.735298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Returning 404 to user: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:47.735298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c290aadf-a62d-40bf-acaf-2c619c4a11d4 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 404 len: 111 microversion: 2.20 time: 0.030153 Oct 09 01:06:47.735298 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 523/1042] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:47 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:06:47.740544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-0b76c4e1-a9e6-4020-8b0a-23a731067c9c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '.version_select at 0x7f726c3118b0>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:47.759232 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-0b76c4e1-a9e6-4020-8b0a-23a731067c9c tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-454303479" status: 204 len: 0 microversion: 2.20 time: 0.023366 Oct 09 01:06:47.759803 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 520/1043] 10.222.0.33 () {60 vars in 1259 bytes} [Sun Oct 9 01:06:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-454303479 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 09 01:06:49.495659 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:49.496152 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:49.502581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:49.503060 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:49.503496 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:49.504198 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:49.504648 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:49.505071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:49.518614 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] HTTP exception thrown: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Request method failure captured: Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 HTTP/1.1 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh1zN0RFejgIlRejGebeuKod3w-XJOgFtBs1rs2AyaEyhIOmlH5SPOmzYiX14QlhxrykcEOZPGPR3baTn6hCjNBhJl6gEE4NxbzyP8nsobszHsmCZ9Ixv6cq69LQ6LeEoSfpt5SjWL79E30Tz4Y9e8xRrsDjNijB2ku6Nq_M4qVItyk Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Openstack-Nova-Api-Version: 2.20 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:06:49.520153 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 56b5d764f0044358ac594a2edd6f4c0f Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.521806 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '406c4931-6ce2-45ec-bea3-b60d04e150c9'} Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:49.522713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.524810 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.526373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Returning 404 to user: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:49.526373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-4c5d0ec3-c118-4648-b7fc-79e056b66c5e tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 404 len: 111 microversion: 2.20 time: 0.029795 Oct 09 01:06:49.526373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 524/1044] 10.222.0.33 () {60 vars in 1280 bytes} [Sun Oct 9 01:06:49 2022] DELETE /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:06:49.531213 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:49.531979 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 406c4931-6ce2-45ec-bea3-b60d04e150c9] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:49.538546 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:49.539652 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:49.540108 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:49.540802 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:49.541262 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:49.541693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:49.554629 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] HTTP exception thrown: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Request method failure captured: Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 HTTP/1.1 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh1zN0RFejgIlRejGebeuKod3w-XJOgFtBs1rs2AyaEyhIOmlH5SPOmzYiX14QlhxrykcEOZPGPR3baTn6hCjNBhJl6gEE4NxbzyP8nsobszHsmCZ9Ixv6cq69LQ6LeEoSfpt5SjWL79E30Tz4Y9e8xRrsDjNijB2ku6Nq_M4qVItyk Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Openstack-Nova-Api-Version: 2.20 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:06:49.556294 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 56b5d764f0044358ac594a2edd6f4c0f Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.557932 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '406c4931-6ce2-45ec-bea3-b60d04e150c9'} Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:49.558751 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. Oct 09 01:06:49.560094 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:49.561166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Returning 404 to user: Instance 406c4931-6ce2-45ec-bea3-b60d04e150c9 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:49.561166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-8796a2c1-c4d9-42d6-a12a-99286146886d tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9" status: 404 len: 111 microversion: 2.20 time: 0.031081 Oct 09 01:06:49.561166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 521/1045] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:49 2022] GET /compute/v2.1/servers/406c4931-6ce2-45ec-bea3-b60d04e150c9 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:06:50.692522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:50.693019 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:50.699895 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:50.700374 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:50.700808 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:50.701484 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:50.701930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:50.702348 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:50.716079 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] HTTP exception thrown: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Request method failure captured: Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 HTTP/1.1 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh1zN0RFejgIlRejGebeuKod3w-XJOgFtBs1rs2AyaEyhIOmlH5SPOmzYiX14QlhxrykcEOZPGPR3baTn6hCjNBhJl6gEE4NxbzyP8nsobszHsmCZ9Ixv6cq69LQ6LeEoSfpt5SjWL79E30Tz4Y9e8xRrsDjNijB2ku6Nq_M4qVItyk Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Openstack-Nova-Api-Version: 2.20 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:06:50.717591 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: 56b5d764f0044358ac594a2edd6f4c0f Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.719245 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '98a78205-334d-469d-8eb4-26c151df1616'} Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:50.720167 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.721397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.722638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Returning 404 to user: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:50.722638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-ffffae28-618f-41c5-846f-aed8af4a8d29 tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 404 len: 111 microversion: 2.20 time: 0.030290 Oct 09 01:06:50.722638 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 525/1046] 10.222.0.33 () {60 vars in 1280 bytes} [Sun Oct 9 01:06:50 2022] DELETE /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:06:50.727813 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:06:50.728419 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] [instance: 98a78205-334d-469d-8eb4-26c151df1616] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:06:50.734695 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:50.735175 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:50.735627 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:50.736313 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:06:50.736762 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:06:50.737210 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:06:50.750253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] HTTP exception thrown: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Request method failure captured: Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 HTTP/1.1 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh1zN0RFejgIlRejGebeuKod3w-XJOgFtBs1rs2AyaEyhIOmlH5SPOmzYiX14QlhxrykcEOZPGPR3baTn6hCjNBhJl6gEE4NxbzyP8nsobszHsmCZ9Ixv6cq69LQ6LeEoSfpt5SjWL79E30Tz4Y9e8xRrsDjNijB2ku6Nq_M4qVItyk Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Openstack-Nova-Api-Version: 2.20 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:06:50.751793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/410212da4a794c07b30ed22cc5b7e760"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: 410212da4a794c07b30ed22cc5b7e760 Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-260510153 Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: 56b5d764f0044358ac594a2edd6f4c0f Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-AttachVolumeShelveTestJSON-260510153-project Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.753527 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '98a78205-334d-469d-8eb4-26c151df1616'} Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:06:50.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. Oct 09 01:06:50.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:06:50.757236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] Returning 404 to user: Instance 98a78205-334d-469d-8eb4-26c151df1616 could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:06:50.757236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-7a64027f-1d6b-407c-8a41-f83de3c533fc tempest-AttachVolumeShelveTestJSON-260510153 tempest-AttachVolumeShelveTestJSON-260510153-project] 10.222.0.33 "GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616" status: 404 len: 111 microversion: 2.20 time: 0.029273 Oct 09 01:06:50.757236 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 522/1047] 10.222.0.33 () {60 vars in 1277 bytes} [Sun Oct 9 01:06:50 2022] GET /compute/v2.1/servers/98a78205-334d-469d-8eb4-26c151df1616 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 09 01:07:00.036821 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-106807095", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "networks": [{"uuid": "7cd2bff9-2f53-480c-b55a-68a046190b11"}]}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:07:00.091269 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] validate_networks() for [('7cd2bff9-2f53-480c-b55a-68a046190b11', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:07:00.439227 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:07:00.439686 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:07:00.440161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:07:00.440567 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:07:00.440976 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:07:00.450214 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:00.450633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:00.451068 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:00.451827 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:00.452283 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:00.452710 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:00.469383 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Getting quotas for project eca3b6a4954f446c872e43cb98c99528. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:07:00.473260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Getting quotas for user f2791fd8579d4906bb7e341c6487f392 and project eca3b6a4954f446c872e43cb98c99528. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:07:00.483963 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:07:00.486202 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:07:00.532287 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-8bcc1e65-6a89-4a9f-960a-c2bd8488fd70 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.499523 Oct 09 01:07:00.532923 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 526/1048] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:07:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 501 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 09 01:07:00.544072 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-e79fa8f4-ef1a-40a6-84e8-9527d0c26825 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:00.544734 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-e79fa8f4-ef1a-40a6-84e8-9527d0c26825 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:00.601453 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-e79fa8f4-ef1a-40a6-84e8-9527d0c26825 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:00.609693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-e79fa8f4-ef1a-40a6-84e8-9527d0c26825 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1218 microversion: 2.1 time: 0.069716 Oct 09 01:07:00.610285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 523/1049] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:00 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1218 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:01.633873 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:01.634418 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:01.640504 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:01.640987 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:01.641417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:01.642113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:01.642791 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:01.643217 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:01.749183 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:01.757051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-2d1666eb-249d-48a6-a4ce-ed60cb5cb835 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1278 microversion: 2.1 time: 0.128069 Oct 09 01:07:01.757855 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 527/1050] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:01 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1278 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:07:02.782050 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:02.782660 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:02.784641 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:02.792798 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:02.793288 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:02.793730 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:02.794431 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:02.794888 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:02.795318 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:02.928407 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:02.936758 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-975be71e-b817-4ba5-b0d6-fadca41b2c03 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.159172 Oct 09 01:07:02.937400 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 524/1051] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:02 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:07:03.341262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-086cb110-8af4-4fd2-9d1b-b4de5cc5b0f9 req-b1ed3ab5-a81a-46f9-b084-483aa90b1191 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2d3659d2-ad76-45be-ab68-eeb1b7edd1cf", "tag": "e75a4675-58d6-4e5d-a818-f3569f47570c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:07:03.351708 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-086cb110-8af4-4fd2-9d1b-b4de5cc5b0f9 req-b1ed3ab5-a81a-46f9-b084-483aa90b1191 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:03.352844 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-086cb110-8af4-4fd2-9d1b-b4de5cc5b0f9 req-b1ed3ab5-a81a-46f9-b084-483aa90b1191 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:03.353522 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-086cb110-8af4-4fd2-9d1b-b4de5cc5b0f9 req-b1ed3ab5-a81a-46f9-b084-483aa90b1191 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:03.364459 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-086cb110-8af4-4fd2-9d1b-b4de5cc5b0f9 req-b1ed3ab5-a81a-46f9-b084-483aa90b1191 service nova] Creating event network-changed:e75a4675-58d6-4e5d-a818-f3569f47570c for instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf on hv-cinder-maste Oct 09 01:07:03.369845 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-086cb110-8af4-4fd2-9d1b-b4de5cc5b0f9 req-b1ed3ab5-a81a-46f9-b084-483aa90b1191 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032221 Oct 09 01:07:03.370439 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 528/1052] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:07:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:07:03.959628 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:03.960272 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:03.967379 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:03.967882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:03.968319 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:03.969004 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:03.969456 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:03.969887 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:04.098308 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:04.105980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2283834b-f311-4959-9c3f-aeffa00a846b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.150286 Oct 09 01:07:04.106537 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 525/1053] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:03 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:05.129417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:05.130105 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:05.136154 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:05.136637 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:05.137071 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:05.137752 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:05.138204 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:05.138630 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:05.269912 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:05.277986 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-98b7e615-d9b9-4c25-b4c0-322911385e77 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.153328 Oct 09 01:07:05.278548 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 529/1054] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:05 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:06.301471 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:06.302129 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:06.309104 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:06.309585 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:06.310021 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:06.310707 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:06.311159 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:06.311584 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:06.437432 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:06.444978 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c17260ba-1b3e-422e-886b-e88af4bb89b1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.147439 Oct 09 01:07:06.445530 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 526/1055] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:06 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:07.467396 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:07.468081 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:07.474218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:07.474700 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:07.475134 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:07.475843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:07.476297 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:07.476719 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:07.607865 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:07.615666 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-c3b2970a-07ee-4190-934d-20b250164114 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.152055 Oct 09 01:07:07.631026 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 530/1056] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:07 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:08.639182 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:08.639850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:08.646034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:08.646520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:08.646959 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:08.647670 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:08.648126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:08.648554 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:08.778429 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:08.786322 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5b2b8ada-8ffb-4f32-ba44-7d22bf844204 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.150971 Oct 09 01:07:08.786885 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 527/1057] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:08 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:09.735996 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-7dc86a49-3e5d-4da8-a49b-034117a26012 req-816a37d4-2d48-4d53-b7eb-8fff0cd666ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2d3659d2-ad76-45be-ab68-eeb1b7edd1cf", "name": "network-vif-plugged", "status": "completed", "tag": "e75a4675-58d6-4e5d-a818-f3569f47570c"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:07:09.745692 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7dc86a49-3e5d-4da8-a49b-034117a26012 req-816a37d4-2d48-4d53-b7eb-8fff0cd666ea service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:09.746164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7dc86a49-3e5d-4da8-a49b-034117a26012 req-816a37d4-2d48-4d53-b7eb-8fff0cd666ea service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:09.746588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7dc86a49-3e5d-4da8-a49b-034117a26012 req-816a37d4-2d48-4d53-b7eb-8fff0cd666ea service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:09.758775 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-7dc86a49-3e5d-4da8-a49b-034117a26012 req-816a37d4-2d48-4d53-b7eb-8fff0cd666ea service nova] Creating event network-vif-plugged:e75a4675-58d6-4e5d-a818-f3569f47570c for instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf on hv-cinder-maste Oct 09 01:07:09.764417 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-7dc86a49-3e5d-4da8-a49b-034117a26012 req-816a37d4-2d48-4d53-b7eb-8fff0cd666ea service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.149692 Oct 09 01:07:09.765141 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 531/1058] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:07:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:07:09.810089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:09.810946 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:09.818012 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:09.818556 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:09.819089 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:09.819915 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:09.820439 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:09.820966 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:09.954328 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:09.969993 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-61999f07-c7b4-462b-b281-0273507f1438 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.164146 Oct 09 01:07:09.970578 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 528/1059] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:09 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 09 01:07:10.990633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:10.991558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:10.996690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:10.997632 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:11.005553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:11.006023 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:11.006454 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:11.007305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:11.007797 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:11.008624 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:11.148436 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:11.156363 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-d0aa474f-c4cb-4c11-a21b-f1043456f41c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1318 microversion: 2.1 time: 0.166741 Oct 09 01:07:11.156919 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 532/1060] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:10 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1318 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:12.178265 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:12.178872 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:12.180783 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:12.186797 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:12.187278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:12.187746 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:12.188465 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:12.188922 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:12.189347 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:12.318557 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:12.326744 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-a83940db-a296-44e2-a326-3fb3f300ba49 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1507 microversion: 2.1 time: 0.152405 Oct 09 01:07:12.327311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 529/1061] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:12 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:26.458076 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4a99b1ec-fa52-4681-9418-7dbed8dd20d9", "device": "/dev/sdb"}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:07:26.461036 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:26.463945 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:26.464709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=54 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:26.470480 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:26.470966 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:26.471407 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:26.472126 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:26.472581 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:26.473012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:26.736511 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1dc93cef-0760-4dbf-8e2a-ee0f4227e89b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "POST /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.282086 Oct 09 01:07:26.737113 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 533/1062] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:07:26 2022] POST /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf/os-volume_attachments => generated 194 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:07:47.341335 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:47.341846 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:47.343904 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:47.344610 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=36 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:47.350871 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:47.351351 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:47.351827 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:47.352523 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:47.352980 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:47.353408 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:47.435476 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.objects.instance [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lazy-loading 'services' on Instance uuid 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf {{(pid=120632) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:07:47.505824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-561f1da2-eee5-4e61-b470-723a0454b386 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "DELETE /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf/os-volume_attachments/4a99b1ec-fa52-4681-9418-7dbed8dd20d9" status: 202 len: 0 microversion: 2.1 time: 0.168537 Oct 09 01:07:47.506428 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 530/1063] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:07:47 2022] DELETE /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf/os-volume_attachments/4a99b1ec-fa52-4681-9418-7dbed8dd20d9 => generated 0 bytes in 170 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:07:48.630509 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:48.631289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:07:48.635767 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:48.636344 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:48.644112 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:48.644598 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:48.645031 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:48.645713 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:48.646163 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:48.646588 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:48.672936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Going to try to terminate instance {{(pid=120633) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:07:48.746781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-f2da5cd0-e6ad-4554-9de3-96ed063bc83e tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "DELETE /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 204 len: 0 microversion: 2.1 time: 0.117503 Oct 09 01:07:48.747376 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 534/1064] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:07:48 2022] DELETE /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:07:48.756022 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:48.756722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:48.763257 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:48.763784 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:48.764230 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:48.764925 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:48.765387 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:48.765818 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:48.894850 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:48.902544 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-c93a655a-9db6-42c7-96d8-646ffa66484c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1513 microversion: 2.1 time: 0.150712 Oct 09 01:07:48.903097 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 531/1065] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:48 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1513 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:49.928028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:49.928712 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:49.934910 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:49.935388 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:49.935847 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:49.936523 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:49.936968 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:49.937382 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:50.065887 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:50.073526 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-cde6bc5b-8d4f-4ee2-bd64-b1074b1082cd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1513 microversion: 2.1 time: 0.149768 Oct 09 01:07:50.074095 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 535/1066] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:49 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1513 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:50.955653 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [req-1cf3ba64-e803-4678-b92f-1279021d0d08 req-ea7c088f-4f27-4a45-bbea-c5db8d6d0d69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2d3659d2-ad76-45be-ab68-eeb1b7edd1cf", "name": "network-vif-deleted", "tag": "e75a4675-58d6-4e5d-a818-f3569f47570c"}]} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:07:50.965859 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-1cf3ba64-e803-4678-b92f-1279021d0d08 req-ea7c088f-4f27-4a45-bbea-c5db8d6d0d69 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:50.966438 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-1cf3ba64-e803-4678-b92f-1279021d0d08 req-ea7c088f-4f27-4a45-bbea-c5db8d6d0d69 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:50.966977 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [req-1cf3ba64-e803-4678-b92f-1279021d0d08 req-ea7c088f-4f27-4a45-bbea-c5db8d6d0d69 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:50.978385 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.compute.server_external_events [req-1cf3ba64-e803-4678-b92f-1279021d0d08 req-ea7c088f-4f27-4a45-bbea-c5db8d6d0d69 service nova] Creating event network-vif-deleted:e75a4675-58d6-4e5d-a818-f3569f47570c for instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf on hv-cinder-maste Oct 09 01:07:50.983741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [req-1cf3ba64-e803-4678-b92f-1279021d0d08 req-ea7c088f-4f27-4a45-bbea-c5db8d6d0d69 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031272 Oct 09 01:07:50.984517 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 532/1067] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:07:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:07:51.096553 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:51.097164 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:51.103645 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:51.104056 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:51.104493 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:51.105180 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:51.105633 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:51.106054 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:51.137565 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:51.138047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:51.138532 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:51.139490 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lazy-loading 'fault' on Instance uuid 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:07:51.180843 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:07:51.188378 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-b3d9d820-1a6a-45a5-a4b5-f9e0359a18e4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 200 len: 1308 microversion: 2.1 time: 0.096542 Oct 09 01:07:51.188930 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 536/1068] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:51 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 1308 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:07:52.212882 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:07:52.213572 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:07:52.220295 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:52.220775 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:52.221212 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:52.221895 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:07:52.222346 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:07:52.222770 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:07:52.237311 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.wsgi [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] HTTP exception thrown: Instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf could not be found. Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Request method failure captured: Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: request: GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf HTTP/1.1 Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept: application/json Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Accept-Encoding: identity Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Connection: close Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Length: 0 Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Content-Type: application/json Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Host: 10.222.0.33 Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Openstack-System-Scope: None Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: User-Agent: python-urllib3/1.26.12 Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Auth-Token: gAAAAABjQh6yw08AhvHPkLUKWId4zRzMLfkCfKCZihYN5VSDXn6yv4e2IhNigYQULW_V4uA_Db_HCv4gmM7TH7_FWyFL_CzkadPxASiGiwm9KsYF4-Eck7dJxTLEMvsr907Gk2cKNpuJ6G2IqiQXe6Dufu0xBdJdpkY28G0tYWPWg1AAUDQBSWg Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Id: None Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Domain-Name: None Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Identity-Status: Confirmed Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Is-Admin-Project: True Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Id: default Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Domain-Name: Default Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Id: eca3b6a4954f446c872e43cb98c99528 Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Project-Name: tempest-VolumesSnapshotTestJSON-732559821 Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Role: member,reader Oct 09 01:07:52.238903 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Roles: member,reader Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant: tempest-VolumesSnapshotTestJSON-732559821 Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Id: eca3b6a4954f446c872e43cb98c99528 Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-732559821 Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User: tempest-VolumesSnapshotTestJSON-732559821-project Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Id: default Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Domain-Name: Default Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Id: f2791fd8579d4906bb7e341c6487f392 Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: X-User-Name: tempest-VolumesSnapshotTestJSON-732559821-project Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: method: > Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: exception: Instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf could not be found. Oct 09 01:07:52.240617 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: action_args: {'id': '2d3659d2-ad76-45be-ab68-eeb1b7edd1cf'} Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf could not be found. Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:07:52.241722 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf could not be found. Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf could not be found. Oct 09 01:07:52.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR nova.api.openstack.wsgi Oct 09 01:07:52.244788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Returning 404 to user: Instance 2d3659d2-ad76-45be-ab68-eeb1b7edd1cf could not be found. {{(pid=120632) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:07:52.244788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ef9ac494-60f6-49b9-9004-55cb3b8ecd29 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf" status: 404 len: 111 microversion: 2.1 time: 0.032156 Oct 09 01:07:52.244788 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 533/1069] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:07:52 2022] GET /compute/v2.1/servers/2d3659d2-ad76-45be-ab68-eeb1b7edd1cf => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:08:00.063661 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-986462069", "imageRef": "bc5ed227-9c8d-48ab-b60a-ccf4833037d9", "flavorRef": "452", "networks": [{"uuid": "7cd2bff9-2f53-480c-b55a-68a046190b11"}]}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:08:00.119874 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] validate_networks() for [('7cd2bff9-2f53-480c-b55a-68a046190b11', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:08:00.458191 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Flavor limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 09 01:08:00.458617 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Image limits 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 09 01:08:00.459084 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Flavor pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 09 01:08:00.459489 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Image pref 0:0:0 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 09 01:08:00.459932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.virt.hardware [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120633) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 09 01:08:00.468837 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:00.469308 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:00.469740 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:00.470455 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:00.470904 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:00.471324 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:00.487440 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Getting quotas for project eca3b6a4954f446c872e43cb98c99528. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:08:00.491190 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.quota [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Getting quotas for user f2791fd8579d4906bb7e341c6487f392 and project eca3b6a4954f446c872e43cb98c99528. Resources: {'cores', 'instances', 'ram'} {{(pid=120633) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:08:00.502360 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Going to run 1 instances... {{(pid=120633) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 09 01:08:00.504631 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bc5ed227-9c8d-48ab-b60a-ccf4833037d9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120633) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 09 01:08:00.550066 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-260f8237-e0ec-4d44-9ded-abdb17a4b24d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.490299 Oct 09 01:08:00.550709 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 537/1070] 10.222.0.33 () {60 vars in 1147 bytes} [Sun Oct 9 01:08:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 492 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 09 01:08:00.562358 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-caeabb0e-73be-4fd9-9896-758e29a9995d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:00.562977 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-caeabb0e-73be-4fd9-9896-758e29a9995d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:00.615015 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-caeabb0e-73be-4fd9-9896-758e29a9995d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:00.621891 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-caeabb0e-73be-4fd9-9896-758e29a9995d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1218 microversion: 2.1 time: 0.064060 Oct 09 01:08:00.622520 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 534/1071] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:00 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1218 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:01.646069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:01.647657 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:01.656179 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:01.656903 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:01.657397 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:01.658305 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:01.658953 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:01.659408 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:01.760449 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:01.767872 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-71f088af-67b7-452e-bde1-facd1b39c541 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1288 microversion: 2.1 time: 0.126810 Oct 09 01:08:01.768425 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 538/1072] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:01 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1288 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:02.791376 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:02.792286 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:02.794969 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:02.803514 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:02.803878 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:02.804337 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:02.805034 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:02.805485 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:02.805905 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:02.912982 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-7fe57e74-2c12-41ec-b4f7-7a5f250cd601 req-3891615a-0320-45b2-8326-e1cdcbc8b603 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "34b6c397-86fd-4b8c-88b8-6f1c06712aae", "tag": "412074d4-7d44-4fcc-aa1a-123e0303dc5f"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:08:02.925010 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7fe57e74-2c12-41ec-b4f7-7a5f250cd601 req-3891615a-0320-45b2-8326-e1cdcbc8b603 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:02.925593 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7fe57e74-2c12-41ec-b4f7-7a5f250cd601 req-3891615a-0320-45b2-8326-e1cdcbc8b603 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:02.926083 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-7fe57e74-2c12-41ec-b4f7-7a5f250cd601 req-3891615a-0320-45b2-8326-e1cdcbc8b603 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:02.940032 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-7fe57e74-2c12-41ec-b4f7-7a5f250cd601 req-3891615a-0320-45b2-8326-e1cdcbc8b603 service nova] Creating event network-changed:412074d4-7d44-4fcc-aa1a-123e0303dc5f for instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae on hv-cinder-maste Oct 09 01:08:02.945453 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-7fe57e74-2c12-41ec-b4f7-7a5f250cd601 req-3891615a-0320-45b2-8326-e1cdcbc8b603 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037105 Oct 09 01:08:02.946690 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 539/1073] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:08:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:08:02.952488 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:02.960152 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-6e33af65-e65f-422b-bc76-af4ff92c61dd tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1318 microversion: 2.1 time: 0.172973 Oct 09 01:08:02.960699 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 535/1074] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:02 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1318 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:03.983850 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:03.984545 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:03.990539 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:03.991021 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:03.991457 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:03.992166 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:03.992620 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:03.993043 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:04.146353 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:04.153964 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-469de7f3-b30f-49e7-a356-8fc170998ab1 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1318 microversion: 2.1 time: 0.174785 Oct 09 01:08:04.155551 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 540/1075] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:03 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1318 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:05.177303 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:05.177947 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:05.184468 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:05.184950 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:05.185392 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:05.186126 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:05.186581 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:05.187007 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:05.461642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:05.469876 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-38bdeb56-4164-4829-89b8-c35e2ffec985 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1318 microversion: 2.1 time: 0.296422 Oct 09 01:08:05.470429 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 536/1076] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:05 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1318 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:06.492836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:06.493478 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:06.499901 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:06.500379 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:06.500813 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:06.501498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:06.501948 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:06.502368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:06.628291 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:06.636027 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-17131299-29f3-44af-93b9-b577dcf27b4b tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1318 microversion: 2.1 time: 0.146953 Oct 09 01:08:06.636599 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 541/1077] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:06 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1318 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:07.659914 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:07.660532 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:07.666858 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:07.667330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:07.667793 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:07.668474 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:07.668919 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:07.669336 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:07.805297 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:07.813227 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-10083075-2f4d-43e9-b3fb-0f3545fe504d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1318 microversion: 2.1 time: 0.158746 Oct 09 01:08:07.814010 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 537/1078] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:07 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:07.894280 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-32ff8fb2-efbc-42ad-9d6f-51a0ffdad6e1 req-8185fa50-675c-4fb9-b75d-ee45c072e28e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34b6c397-86fd-4b8c-88b8-6f1c06712aae", "name": "network-vif-plugged", "status": "completed", "tag": "412074d4-7d44-4fcc-aa1a-123e0303dc5f"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:08:07.905781 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-32ff8fb2-efbc-42ad-9d6f-51a0ffdad6e1 req-8185fa50-675c-4fb9-b75d-ee45c072e28e service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:07.906262 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-32ff8fb2-efbc-42ad-9d6f-51a0ffdad6e1 req-8185fa50-675c-4fb9-b75d-ee45c072e28e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:07.906694 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-32ff8fb2-efbc-42ad-9d6f-51a0ffdad6e1 req-8185fa50-675c-4fb9-b75d-ee45c072e28e service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:07.917973 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-32ff8fb2-efbc-42ad-9d6f-51a0ffdad6e1 req-8185fa50-675c-4fb9-b75d-ee45c072e28e service nova] Creating event network-vif-plugged:412074d4-7d44-4fcc-aa1a-123e0303dc5f for instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae on hv-cinder-maste Oct 09 01:08:07.923289 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-32ff8fb2-efbc-42ad-9d6f-51a0ffdad6e1 req-8185fa50-675c-4fb9-b75d-ee45c072e28e service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032663 Oct 09 01:08:07.923936 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 542/1079] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:08:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:08:08.837542 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:08.838200 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:08.844642 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:08.845123 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:08.845560 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:08.846242 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:08.846693 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:08.847118 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:08.974316 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:08.982166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-ff4a8618-6aaa-423f-aaf6-61a6186b3724 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1318 microversion: 2.1 time: 0.148414 Oct 09 01:08:08.982731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 538/1080] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:08 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:10.006368 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:10.007181 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:10.013212 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:10.013683 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:10.014110 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:10.014790 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:10.015260 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:10.015706 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:10.141741 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:10.149342 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-9ed856a8-a828-4c43-8852-7bd682a8de3d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1508 microversion: 2.1 time: 0.147723 Oct 09 01:08:10.149913 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 543/1081] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:10 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1508 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:10.172608 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "69080c16-a0ed-4c00-98e3-6d99fb436d81", "device": "/dev/sdb"}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:08:10.175500 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:10.181808 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:10.182285 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:10.182721 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:10.183404 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:10.183954 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:10.184382 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:10.430124 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-cd96e06a-c25c-4d65-af2c-4cf4961ae2d4 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "POST /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.261841 Oct 09 01:08:10.430753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 539/1082] 10.222.0.33 () {60 vars in 1323 bytes} [Sun Oct 9 01:08:10 2022] POST /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae/os-volume_attachments => generated 194 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:08:27.738219 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:27.738742 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:27.740699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:27.741386 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell0:SELECT=6 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:27.743022 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:27.744012 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:27.753059 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:27.753913 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:27.754347 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:27.755047 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:27.755525 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:27.755958 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:27.844926 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.objects.instance [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lazy-loading 'services' on Instance uuid 34b6c397-86fd-4b8c-88b8-6f1c06712aae {{(pid=120633) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 09 01:08:27.916051 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-0f4a427f-8d9f-48b3-b563-17e14384326c tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "DELETE /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae/os-volume_attachments/69080c16-a0ed-4c00-98e3-6d99fb436d81" status: 202 len: 0 microversion: 2.1 time: 0.181513 Oct 09 01:08:27.916814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 544/1083] 10.222.0.33 () {58 vars in 1416 bytes} [Sun Oct 9 01:08:27 2022] DELETE /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae/os-volume_attachments/69080c16-a0ed-4c00-98e3-6d99fb436d81 => generated 0 bytes in 183 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 09 01:08:29.039757 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=2 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:29.040605 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=22 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:29.041253 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=35,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:08:29.046184 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:29.046764 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:29.054897 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:29.055393 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:29.055862 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:29.056571 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:29.057029 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:29.057455 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:29.085278 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Going to try to terminate instance {{(pid=120632) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 09 01:08:29.161741 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-b3741f93-1a54-4c35-bb71-709f9670b343 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "DELETE /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 204 len: 0 microversion: 2.1 time: 0.123105 Oct 09 01:08:29.162302 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 540/1084] 10.222.0.33 () {58 vars in 1239 bytes} [Sun Oct 9 01:08:29 2022] DELETE /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 09 01:08:29.172406 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:29.173296 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:29.179877 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:29.180356 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:29.180788 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:29.181470 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:29.181921 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:29.182339 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:29.315534 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.policy [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120633) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:29.323820 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-1fe4eb76-9cb1-43f9-b883-24ec3bb3a3dc tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1514 microversion: 2.1 time: 0.156763 Oct 09 01:08:29.324384 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 545/1085] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:29 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:30.348068 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:30.348678 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:30.355056 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:30.355541 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:30.356017 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:30.356709 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:30.357166 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:30.358086 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:30.486246 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:30.493926 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-107d4e7d-4495-4d73-aaf4-ec976461bdb6 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1514 microversion: 2.1 time: 0.150181 Oct 09 01:08:30.494496 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 541/1086] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:30 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1514 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:31.465228 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [req-8f98d7f8-5290-4c78-8685-f264bb6001d5 req-d385b0e6-160f-4172-b6f4-537f7bab3552 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34b6c397-86fd-4b8c-88b8-6f1c06712aae", "name": "network-vif-deleted", "tag": "412074d4-7d44-4fcc-aa1a-123e0303dc5f"}]} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:08:31.479069 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-8f98d7f8-5290-4c78-8685-f264bb6001d5 req-d385b0e6-160f-4172-b6f4-537f7bab3552 service nova] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:31.479558 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-8f98d7f8-5290-4c78-8685-f264bb6001d5 req-d385b0e6-160f-4172-b6f4-537f7bab3552 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:31.480161 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [req-8f98d7f8-5290-4c78-8685-f264bb6001d5 req-d385b0e6-160f-4172-b6f4-537f7bab3552 service nova] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:31.492934 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.compute.server_external_events [req-8f98d7f8-5290-4c78-8685-f264bb6001d5 req-d385b0e6-160f-4172-b6f4-537f7bab3552 service nova] Creating event network-vif-deleted:412074d4-7d44-4fcc-aa1a-123e0303dc5f for instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae on hv-cinder-maste Oct 09 01:08:31.498211 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [req-8f98d7f8-5290-4c78-8685-f264bb6001d5 req-d385b0e6-160f-4172-b6f4-537f7bab3552 service nova] 10.222.0.33 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036989 Oct 09 01:08:31.498836 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 546/1087] 10.222.0.33 () {64 vars in 1317 bytes} [Sun Oct 9 01:08:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:08:31.515979 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:31.516590 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.compute.api [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120632) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:31.524252 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:31.524731 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:31.525186 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:31.525879 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:31.526330 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:31.526753 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG oslo_concurrency.lockutils [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120632) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:31.594723 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.policy [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2791fd8579d4906bb7e341c6487f392', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eca3b6a4954f446c872e43cb98c99528', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120632) authorize /opt/stack/nova/nova/policy.py:199}} Oct 09 01:08:31.602824 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-5556c776-61c4-4f3c-98fa-7e1f61020b4d tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 200 len: 1303 microversion: 2.1 time: 0.090741 Oct 09 01:08:31.603386 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 542/1088] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:31 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 1303 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 09 01:08:32.627276 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Calling method '>' {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:08:32.627932 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.compute.api [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] [instance: 34b6c397-86fd-4b8c-88b8-6f1c06712aae] Fetching instance by UUID {{(pid=120633) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 09 01:08:32.634169 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:32.634653 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:32.635088 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:32.635814 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Acquiring lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 09 01:08:32.636273 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 09 01:08:32.636699 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG oslo_concurrency.lockutils [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Lock "8734a8b3-2f61-4ffd-a611-f81fb26e2dd6" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120633) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 09 01:08:32.650006 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.wsgi [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] HTTP exception thrown: Instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae could not be found. Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Request method failure captured: Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: request: GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae HTTP/1.1 Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept: application/json Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Accept-Encoding: identity Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Connection: close Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Length: 0 Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Content-Type: application/json Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Host: 10.222.0.33 Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Openstack-System-Scope: None Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: User-Agent: python-urllib3/1.26.12 Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Auth-Token: gAAAAABjQh6yw08AhvHPkLUKWId4zRzMLfkCfKCZihYN5VSDXn6yv4e2IhNigYQULW_V4uA_Db_HCv4gmM7TH7_FWyFL_CzkadPxASiGiwm9KsYF4-Eck7dJxTLEMvsr907Gk2cKNpuJ6G2IqiQXe6Dufu0xBdJdpkY28G0tYWPWg1AAUDQBSWg Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Id: None Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Domain-Name: None Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Identity-Status: Confirmed Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Is-Admin-Project: True Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Id: default Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Domain-Name: Default Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Id: eca3b6a4954f446c872e43cb98c99528 Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Project-Name: tempest-VolumesSnapshotTestJSON-732559821 Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Role: member,reader Oct 09 01:08:32.651618 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Roles: member,reader Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.33:8080", "publicURL": "http://10.222.0.33:8080/v1/AUTH_eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/volume/v3/eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/image"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/compute/v2/eca3b6a4954f446c872e43cb98c99528"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.33/identity"}]}] Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant: tempest-VolumesSnapshotTestJSON-732559821 Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Id: eca3b6a4954f446c872e43cb98c99528 Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-732559821 Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User: tempest-VolumesSnapshotTestJSON-732559821-project Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Id: default Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Domain-Name: Default Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Id: f2791fd8579d4906bb7e341c6487f392 Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: X-User-Name: tempest-VolumesSnapshotTestJSON-732559821-project Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: method: > Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: exception: Instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae could not be found. Oct 09 01:08:32.653366 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: action_args: {'id': '34b6c397-86fd-4b8c-88b8-6f1c06712aae'} Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae could not be found. Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 09 01:08:32.654882 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae could not be found. Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae could not be found. Oct 09 01:08:32.656826 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: ERROR nova.api.openstack.wsgi Oct 09 01:08:32.658373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] Returning 404 to user: Instance 34b6c397-86fd-4b8c-88b8-6f1c06712aae could not be found. {{(pid=120633) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 09 01:08:32.658373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO nova.api.openstack.requestlog [None req-458a1760-9c2b-4d05-9c72-3463b07e1779 tempest-VolumesSnapshotTestJSON-732559821 tempest-VolumesSnapshotTestJSON-732559821-project] 10.222.0.33 "GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae" status: 404 len: 111 microversion: 2.1 time: 0.031068 Oct 09 01:08:32.658373 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: [pid: 120633|app: 0|req: 547/1089] 10.222.0.33 () {58 vars in 1236 bytes} [Sun Oct 9 01:08:32 2022] GET /compute/v2.1/servers/34b6c397-86fd-4b8c-88b8-6f1c06712aae => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 09 01:09:04.548743 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:09:04.549495 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:09:04.713817 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-2464b8cc-02c6-4f60-b8ad-675edfe599e0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] Action: 'create', calling method: .version_select at 0x7f72689308b0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1881940595", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJMGLbODZzqHlI0skhasy59/eV5NAaOKpbVIUiiVxQmaLBEgwfLD8mF3L53fdz6Bh59b24F09AM6GX4Q6ofYr3tAPhCVuO4PrfIgkJyJzXXwzvsykI+X8LtyKY8HoavkJw=="}} {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:09:04.730085 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-2464b8cc-02c6-4f60-b8ad-675edfe599e0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] Getting quotas for project 1fef9408db82475ab8903f48e483dd1b. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 09 01:09:04.734028 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.quota [None req-2464b8cc-02c6-4f60-b8ad-675edfe599e0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] Getting quotas for user 72bdabb024674f48927b902d2225e473 and project 1fef9408db82475ab8903f48e483dd1b. Resources: {'key_pairs'} {{(pid=120632) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 09 01:09:04.746158 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: ERROR oslo.messaging._drivers.impl_rabbit [None req-2464b8cc-02c6-4f60-b8ad-675edfe599e0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] [3a19c9a7-d06b-447e-9d5b-e3b2fc323ff9] AMQP server on 10.222.0.33:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Oct 09 01:09:05.753844 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO oslo.messaging._drivers.impl_rabbit [None req-2464b8cc-02c6-4f60-b8ad-675edfe599e0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] [3a19c9a7-d06b-447e-9d5b-e3b2fc323ff9] Reconnected to AMQP server on 10.222.0.33:5672 via [amqp] client with port 50538. Oct 09 01:09:05.771417 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-2464b8cc-02c6-4f60-b8ad-675edfe599e0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] 10.222.0.33 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 1.228138 Oct 09 01:09:05.772250 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 543/1090] 10.222.0.33 () {60 vars in 1159 bytes} [Sun Oct 9 01:09:04 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 1229 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 09 01:09:20.946230 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.api.openstack.wsgi [None req-500f124c-2089-41dd-9e3f-cd2b3dffabf5 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-169118623", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1881940595", "security_groups": [{"name": "tempest-secgroup-smoke-1763834362"}], "networks": [{"uuid": "b61dd093-5d2d-4770-aeef-41797121c0d3"}], "block_device_mapping_v2": [{"uuid": "d1cec9c3-19e4-4aba-be43-4491e6336106", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120633) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 09 01:09:20.956675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=8 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:09:20.957318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:09:21.060137 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG nova.network.neutron [None req-500f124c-2089-41dd-9e3f-cd2b3dffabf5 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] validate_networks() for [('b61dd093-5d2d-4770-aeef-41797121c0d3', None, None, None, None, None)] {{(pid=120633) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 09 01:09:30.961466 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: DEBUG dbcounter [-] [120633] Writing DB stats nova_api:SELECT=2 {{(pid=120633) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:10:21.200797 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_cell0:SELECT=4 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:10:21.201867 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG dbcounter [-] [120632] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=120632) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 09 01:10:21.203393 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Oct 09 01:10:21.203616 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: Sun Oct 9 01:10:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1881940595 (ip 10.222.0.33) !!! Oct 09 01:10:21.212603 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: DEBUG nova.api.openstack.wsgi [None req-84684bd0-b9da-4937-b40a-08357c4442b0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] Calling method '.version_select at 0x7f72688fc820>' {{(pid=120632) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 09 01:10:21.236567 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: INFO nova.api.openstack.requestlog [None req-84684bd0-b9da-4937-b40a-08357c4442b0 tempest-TestShelveInstance-710885321 tempest-TestShelveInstance-710885321-project] 10.222.0.33 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1881940595" status: 202 len: 0 microversion: 2.1 time: 0.037596 Oct 09 01:10:21.237137 dv-cinder-master-3-iscsi devstack@n-api.service[120632]: [pid: 120632|app: 0|req: 544/1091] 10.222.0.33 () {58 vars in 1254 bytes} [Sun Oct 9 01:10:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1881940595 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 09 01:11:08.642218 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 09 01:11:08.642834 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Sun Oct 9 01:11:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.33) !!! Oct 09 01:11:20.077241 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 09 01:11:20.079430 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Sun Oct 9 01:11:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.33) !!! Oct 09 01:11:47.751028 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 09 01:11:47.751675 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Sun Oct 9 01:11:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.33) !!! Oct 09 01:14:28.652608 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 09 01:14:28.653318 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Sun Oct 9 01:14:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.33) !!! Oct 09 01:14:40.089906 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 09 01:14:40.090498 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Sun Oct 9 01:14:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.33) !!! Oct 09 01:15:07.760459 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 09 01:15:07.761184 dv-cinder-master-3-iscsi devstack@n-api.service[120633]: Sun Oct 9 01:15:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.33) !!!